Ship-shore data out-of-order synchronization method and device, electronic equipment and storage medium

Abstract

The invention discloses a ship-shore data out-of-order synchronization method and device, electronic equipment and a storage medium, and the method comprises the steps: sending the compression information of a to-be-synchronized compression package to a shore end through a data synchronization request sent to the shore end, so as to enable the shore end to carry out the data synchronization detection based on the compression information, and obtaining a full bitmap of the data synchronization condition of the shore end; performing breakpoint detection on the data blocks based on the total bitmap to obtain breakpoint data; when the number of the breakpoint data is greater than the first number, retransmitting the breakpoint data to the shore end; otherwise, synchronizing the data blocks after the target block to the shore end in sequence; periodically acquiring a block receiving condition of a shore end, and updating the total bitmap based on the block receiving condition; and returning to execute the step of carrying out breakpoint detection on the data blocks based on the total bitmap to obtain breakpoint data, and carrying out a new round of data synchronization iteration until all the data blocks of the compressed packet are synchronized. The method can efficiently realize ship-shore data synchronization, and can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, electronic equipment and storage medium for out-of-order synchronization of ship-shore data. Background Art

[0002] In the current ship-to-shore data synchronization technology, the mainstream solution mainly relies on TCP (Transmission Control Protocol) to achieve it. This solution is consistent with the idea of ​​data synchronization between two points on land, that is, first establish a TCP communication channel between the two points, and then start sending synchronization data through the channel. However, there is a significant problem with the existing technology: since the data is sent and retransmitted sequentially, a lot of data that has been successfully received by the receiving end will still be sent repeatedly. This repeated transmission leads to a large amount of retransmitted data, which in turn affects the overall transmission speed. Therefore, the current technology still needs to make further improvements and optimizations in improving transmission efficiency. Summary of the invention

[0003] The present invention aims to solve the problem of related technical limitations at least to a certain extent. To this end, the present invention proposes a method, device, electronic device and storage medium for out-of-order synchronization of ship and shore data, which can efficiently achieve ship and shore data synchronization.

[0004] On the one hand, an embodiment of the present invention provides a method for out-of-order synchronization of ship-shore data, which is applied to a ship side, and the method includes the following steps:

[0005] A data synchronization request is initiated to the shore end, and the compression information of the compressed package to be synchronized is sent to the shore end, so that the shore end performs data synchronization detection based on the compression information and obtains a full bitmap of the data synchronization status of the shore end;

[0006] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0007] Perform breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the amount of breakpoint data is greater than the first amount, resend the breakpoint data to the shore end; otherwise, synchronize the data block sequence after the target block to the shore end;

[0008] The synchronization state of the breakpoint data is in sync, and the breakpoint data indicates that there is a data block with a synchronization state of completed synchronization behind it; the target block indicates the data block with the largest sequence number with a synchronization state of completed synchronization;

[0009] Regularly obtain the block reception status of the shore end, and update the full bitmap based on the block reception status;

[0010] Return to the step of performing breakpoint detection on the data blocks based on the full bitmap to obtain the breakpoint data, and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0011] Optionally, periodically obtaining the block receiving status of the shore end includes at least one of the following steps:

[0012] When the number of data blocks received by the shore end in a round of data synchronization iteration is greater than the second number, obtaining the block reception status of the shore end in response to a reply from the shore end;

[0013] Block verification requests and heartbeat requests are sent to the shore end regularly, so that the shore end replies with the block reception status in response to the block verification request and / or heartbeat request, and then the block reception status of the shore end is obtained regularly.

[0014] Optionally, the block reception situation includes a bitmap start ID and incremental bitmap information, the bitmap start ID corresponds to the largest sequence number of all data blocks that have completed data synchronization; the bitmap start ID represents the sequence number corresponding to the start bitmap, the synchronization state of the start bitmap is that synchronization is completed, and the start bitmap represents that there is no data block with the largest sequence number in the synchronization state before; updating the full bitmap based on the block reception situation includes the following steps:

[0015] Update the synchronization status of the start bitmap and all the data blocks before it to complete synchronization;

[0016] The incremental bitmap information includes the synchronization status of a preset number of data blocks following the data block corresponding to the start bitmap;

[0017] Based on the incremental bitmap information, the synchronization status of a preset number of data blocks following the start bitmap is updated;

[0018] The status update includes updating the synchronization status of the data blocks to be transmitted in the next round of data synchronization iteration from waiting for synchronization to synchronizing.

[0019] On the other hand, an embodiment of the present invention provides a ship-shore data out-of-order synchronization device, which is applied to a ship side, and includes:

[0020] The first module is used to initiate a data synchronization request to the shore end, and send the compressed information of the compressed package to be synchronized to the shore end, so that the shore end performs data synchronization detection based on the compressed information and obtains a full bitmap of the data synchronization status of the shore end;

[0021] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0022] The second module is used to perform breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the amount of breakpoint data is greater than the first amount, the breakpoint data is resent to the shore end; otherwise, the data block sequence after the target block is synchronized to the shore end;

[0023] The synchronization state of the breakpoint data is in sync, and the breakpoint data indicates that there is a data block with a synchronization state of completed synchronization behind it; the target block indicates the data block with the largest sequence number with a synchronization state of completed synchronization;

[0024] The third module is used to periodically obtain the block reception status of the shore end and update the full bitmap based on the block reception status;

[0025] The fourth module is used to return to execute the second module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0026] On the other hand, an embodiment of the present invention provides a method for synchronizing ship-shore data out of order, which is applied to the shore side. The method is interactively applied with the aforementioned method for synchronizing ship-shore data out of order. The method includes the following steps:

[0027] In response to a data synchronization request from the ship side, the compression information of the compressed package to be synchronized is obtained; data synchronization detection is performed based on the compression information to obtain a full bitmap of the data synchronization status; and the full bitmap is sent to the ship side;

[0028] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0029] The data blocks sent by the ship are received out of order, and the received data blocks are stored sequentially based on the sequence numbers;

[0030] Based on the storage status of the data blocks, the block reception status is regularly replied to the ship side, so that the ship side updates the full data bitmap on the ship side based on the block reception status;

[0031] Return to the step of receiving the data blocks sent by the ship in disorder, and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

[0032] Optionally, the ship end periodically sends a block verification request and / or a heartbeat request to the shore end, and periodically replies to the ship end with block reception status, including at least one of the following steps:

[0033] When the number of data blocks received in a round of data synchronization iteration is greater than the second number, the block reception status is replied to the ship end;

[0034] In response to the block verification request and / or heartbeat request periodically sent by the ship side, the block reception status is replied to the ship side.

[0035] Optionally, the ship end periodically sends a block verification request to the shore end, and the method further includes the following steps:

[0036] In response to a block check request from the ship side, obtaining a first check value of the data block sent by the ship side;

[0037] Obtaining a second check value of the received data block by processing with a preset information digest algorithm;

[0038] Performing data consistency verification based on a first verification value and a second verification value corresponding to each synchronized data block;

[0039] When the data consistency check fails, a retransmission request of the corresponding data block is sent to the ship end, so that the ship end retransmits the corresponding data block in response to the retransmission request.

[0040] On the other hand, an embodiment of the present invention provides a ship-shore data out-of-order synchronization device, which is interactively applied with the aforementioned ship-shore data out-of-order synchronization device, and includes:

[0041] The fifth module is used to respond to the data synchronization request of the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization situation; and send the full bitmap to the ship end;

[0042] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0043] The sixth module is used to receive the data blocks sent by the ship side out of order, and store the received data blocks in order based on the sequence number;

[0044] The seventh module is used to periodically reply the block reception status to the ship end based on the storage status of the data blocks, so that the ship end updates the full bitmap of the ship end based on the block reception status;

[0045] The eighth module is used to return to execute the sixth module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0046] Optionally, the ship end periodically sends a block verification request to the shore end, and the device further includes:

[0047] A ninth module is configured to obtain, in response to a block check request from the ship side, a first check value of a data block sent by the ship side;

[0048] A tenth module is used to obtain a second check value of the received data block by processing through a preset information summary algorithm;

[0049] An eleventh module is used to perform data consistency verification based on a first verification value and a second verification value corresponding to each synchronized data block;

[0050] The twelfth module is used to send a retransmission request of the corresponding data block to the ship side when the data consistency check fails, so that the ship side retransmits the corresponding data block in response to the retransmission request.

[0051] On the other hand, an embodiment of the present invention provides an electronic device, including: a processor and a memory; the memory is used to store programs; the processor executes the program to implement the above-mentioned ship-shore data out-of-order synchronization method.

[0052] On the other hand, an embodiment of the present invention provides a computer storage medium, in which a program executable by a processor is stored. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned ship-shore data out-of-order synchronization method.

[0053] The embodiment of the present invention sends the compression information of the compressed package to be synchronized to the shore end through a data synchronization request initiated to the shore end, so that the shore end performs data synchronization detection based on the compression information and obtains a full bitmap of the data synchronization status of the shore end; wherein the full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, in synchronization and waiting for synchronization; breakpoint detection is performed on the data block based on the full bitmap to obtain breakpoint data; when the number of breakpoint data is greater than the first number, the breakpoint data is resent to the shore end; otherwise, the data blocks after the target block are synchronized to the shore end in sequence; wherein the synchronization status of the breakpoint data is in synchronization, and the breakpoint data indicates that there is a data block with a synchronization status of completed synchronization behind it; the target block indicates the data block with the largest sequence number with a synchronization status of completed synchronization; the block reception status of the shore end is regularly obtained, and the full bitmap is updated based on the block reception status; the step of performing breakpoint detection on the data block based on the full bitmap to obtain breakpoint data is returned to perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized. The embodiments of the present invention have at least the following beneficial effects:

[0054] Improve synchronization efficiency: By sending compressed information of the compressed package to the shore end, the shore end can perform data synchronization detection based on the information, thereby reducing unnecessary data transmission and improving synchronization efficiency.

[0055] Breakpoint resumable transmission capability: The solution performs breakpoint detection on data blocks. When the amount of breakpoint data is greater than the first amount, only the breakpoint data is resent to the shore end, avoiding repeated transmission of a large amount of synchronized data and further improving synchronization efficiency.

[0056] Intelligent synchronization strategy: Dynamically adjust the synchronization strategy according to the synchronization status of the data blocks in the full bitmap. For the data blocks that have completed synchronization, they will no longer be transmitted repeatedly, but the data blocks after the target block will be synchronized to the shore in sequence, realizing an intelligent synchronization process.

[0057] Real-time update and iteration: Regularly obtain the block reception status of the shore end, and update the full bitmap based on the status, then return to execute the breakpoint detection step and perform a new round of data synchronization iteration. This real-time update and iteration mechanism ensures the continuity and accuracy of the synchronization process.

[0058] Comprehensive synchronization status monitoring: The full bitmap contains the sequence number and synchronization status of each data block of the compressed package, making the monitoring of the synchronization process more comprehensive and detailed, helping to discover and solve problems in a timely manner.

[0059] In summary, this technical solution significantly improves the performance and reliability of ship-to-shore data synchronization by improving synchronization efficiency, achieving breakpoint resumption, adopting intelligent synchronization strategies, real-time updates and iterations, and comprehensive synchronization status monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0061] Figure 1 A schematic diagram of an implementation environment for out-of-order synchronization of ship-shore data provided by an embodiment of the present invention;

[0062] Figure 2 A schematic flow chart of a method for out-of-order synchronization of ship-shore data applied to a ship side provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of the principle of a full bitmap for synchronous interaction of breakpoint data provided by an embodiment of the present invention;

[0064] Figure 4 A schematic flow chart of a method for out-of-order synchronization of ship-shore data applied to a shore terminal provided by an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the principle flow of conventional ship-shore data synchronization interaction;

[0066] Figure 6 A schematic diagram of an extended process of a method for out-of-order synchronization of ship-shore data applied to a shore end provided in an embodiment of the present invention;

[0067] Figure 7A schematic diagram of the overall business process principle of the ship-shore data out-of-order synchronization method provided by an embodiment of the present invention;

[0068] Figure 8 A schematic diagram of the structure of a ship-shore data out-of-sequence synchronization device applied to a ship side provided by an embodiment of the present invention;

[0069] Fig. 9 A schematic diagram of the structure of a ship-shore data out-of-order synchronization device applied to a shore side provided by an embodiment of the present invention;

[0070] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0072] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100", "second / S200", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0073] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present invention may be combined with other embodiments.

[0074] It is understandable that the ship-shore data out-of-order synchronization method provided in the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this.

[0075] To facilitate the understanding of the technical solution of the present invention, the specific terms of the technical features that may appear in the embodiments of the present invention are first explained:

[0076] VSAT satellite communication (Very Small Aperture Terminal): Since VSAT is derived from the traditional satellite communication system, it is also called a small satellite data station or a personal earth station. The "small" here refers to the small antenna aperture of the small station equipment in the VSAT system, which is usually 0.3m to 1.4m. The equipment has a compact structure, is solid, intelligent, cheap, easy to install, has low requirements on the use environment, is not restricted by the ground network, and has flexible networking.

[0077] A drilling vessel is a ship dedicated to drilling submarine geological structures, mainly used for marine geological exploration. With the in-depth implementation of the national marine strategy, my country's drilling vessels are appearing more and more frequently in international oceans such as the Pacific Ocean. At this time, the only means of communication between the ship and the shore base is satellite.

[0078] TCP Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP is designed to adapt to a layered protocol hierarchy that supports multi-network applications. Pairs of processes in host computers connected to different but interconnected computer communication networks rely on TCP to provide reliable communication services. TCP assumes that it can obtain simple, possibly unreliable datagram services from lower-level protocols. In principle, TCP should be able to operate on a variety of communication systems from hard-wired connections to packet switching or circuit switching networks.

[0079] The UDP transmission control protocol (User Datagram Protocol) is based on a connectionless, unreliable datagram transmission mechanism, and only provides source port, destination port, data length, checksum and encapsulation to achieve fast and concise end-to-end communication. It does not guarantee the orderly arrival of data, reliable transmission or flow control, and is suitable for application scenarios that are sensitive to delay, can tolerate packet loss, and have self-correction capabilities, such as real-time audio and video streaming, online games, network broadcasting, DNS queries, etc.

[0080] LZMA compression algorithm: Among the actual compression algorithms, LZMA (Lempel-Ziv-Markov chain Algorithm) is generally considered to provide extremely high compression rates. It was proposed in 1998 and is an improved version of the LZ77 algorithm, used to implement the 7-Zip file format (.7z). LZMA uses a chain compression method and applies a modified LZ77 algorithm at the bit level to achieve excellent compression efficiency, especially when processing highly repetitive data such as text and source code.

[0081] LZMA2 compression algorithm: As a successor to LZMA, LZMA2 further improves compression performance and multi-core processor support while maintaining a high compression rate. In practical applications, such as using 7-Zip software, choosing the LZMA2 compression method can often achieve a very high compression rate.

[0082] MD5 (Message-Digest Algorithm 5): It is a widely used cryptographic hash function designed to convert information of any length into a fixed-length hash value. This process is one-way, which means that it is difficult to infer the original information from the hash value. The core significance of MD5 is to provide a data verification mechanism. By calculating the MD5 value of the data, it is possible to efficiently compare whether the data is consistent without directly comparing the data itself. It is achieved through complex mathematical operations, and even a small change in the data will result in a huge difference in the calculated MD5 value.

[0083] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the invention. Figure 1 The implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to a network wirelessly or wired to complete data transmission and exchange.

[0084] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0085] In addition, the server 101 can also be a node server in the blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0086] The terminal 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal 102 and the server 101 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention.

[0087] Based on the example Figure 1 In the implementation environment shown, an embodiment of the present invention provides a method for out-of-order synchronization of ship-shore data. The following is explained using the example of the method for out-of-order synchronization of ship-shore data applied in the server 101. It can be understood that the method for out-of-order synchronization of ship-shore data can also be applied in the terminal 102.

[0088] Reference Figure 2 , Figure 2 The flowchart of the method for synchronizing ship-shore data out of order and applied to a server provided in an embodiment of the present invention is provided. The execution subject of the method for synchronizing ship-shore data out of order and applied to a server can be any of the aforementioned computer devices (including a server or a terminal). Figure 2 , the method is applied to the ship side and comprises the following steps:

[0089] S100, a data synchronization request is initiated to the shore end, and the compression information of the compressed package to be synchronized is sent to the shore end, so that the shore end performs data synchronization detection based on the compression information, and obtains a full bitmap of the data synchronization status of the shore end;

[0090] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0091] Exemplarily, in some specific implementations, the ship sends a data synchronization request, which is forwarded to the shore via satellite. The request mainly includes detailed information on a compressed package of transmitted data, including the file name, file size, file MD5, file creation time, last modification time, and other information of the compressed package. This information is mainly used to determine whether breakpoint resumption is required.

[0092] When the shore side receives a data synchronization request, it will check the files that have been transferred by the shore side. The judgment process is mainly as follows:

[0093] 1) Check the file name to see if the file exists on the shore. If it does not exist, it means that the file is being transmitted for the first time. If it does not exist, continue to compare the file's MD5;

[0094] 2) Check the file MD5 to see if the MD5 of the file on the shore is consistent with the MD5 of the target file. If they are inconsistent, it means that the target file has changed and needs to be retransmitted. If they are consistent, continue to compare the file sizes.

[0095] 3) Check the bitmap information and file size. If they are consistent, it means the file has been transferred before and you can choose to resume the transfer. If they are inconsistent, it means the target file has changed and needs to be transferred again. This step is necessary because the file is stored in the cache before it is successfully transferred, and the cache may be tampered with or cleared.

[0096] 4) Check the bitmap information. If all data packets have been received, it means that the data synchronization is successful. Otherwise, it needs to be retransmitted or retransmitted from the breakpoint. If the data has been transmitted successfully, check the final verification mark on the shore side. If the verification has been completed, reply that the ship side synchronization is successful. Otherwise, reply that the final verification is required.

[0097] After the inspection is completed, the monitored situation will be directly fed back to the ship, mainly the existing file information on the shore (file name, MD5, file bitmap, file size, last modified time, transfer progress, etc.). Note that the bitmap here is a full bitmap, that is, the entire bitmap needs to be synchronized to the ship.

[0098] It should be noted that, in some embodiments, before data synchronization, the method may further include a step of locally compressing the data of the file to be transmitted; illustratively, in some specific implementations, the specific implementation may be as follows:

[0099] There are usually many files to be transmitted on the ship side, mainly the test results of technical experiments, in formats including tables, texts, pictures and other files, and fewer high-definition videos. These data files need to be compressed before transmission. This article uses the open source LZMA2 compression algorithm, which has a very high compression rate. Compared with other compression algorithms, the compression speed will be slightly slower, but considering the satellite bandwidth, a higher compression rate is still necessary. In the data transmission management regulations on board, it is not recommended to transmit high-definition video. This is because current videos are usually in compressed format. Even if LZMA2 is used for re-compression, the file compression rate will not be too high, and the video will occupy a lot of bandwidth and affect the transmission of other files, so high-definition video files are usually synchronized by direct copying after the ship docks.

[0100] Under normal circumstances, there will be multiple files to be synchronized. These files are sorted according to the last modification time, and the files will be divided into multiple groups for compression. This article uses 100M as the benchmark for grouping, that is, the sum of the file sizes of each group is around 100M. If a single file exceeds 100M, then this file is grouped separately. Then compression is performed for each group, and the compressed package name is the MD5 of all file names added together, so that a series of compressed packages will be obtained. This grouping standard can be customized, but it is not recommended to exceed 200M, because the larger the compressed file, the higher the file transfer failure rate. These compressed packages are transmitted synchronously one by one, and the present invention is aimed at single compressed packages for data synchronization.

[0101] S200, performing breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the amount of breakpoint data is greater than the first amount, resending the breakpoint data to the shore end; otherwise, synchronizing the data blocks after the target block to the shore end in sequence;

[0102] The synchronization state of the breakpoint data is in sync, and the breakpoint data indicates that there is a data block with a synchronization state of completed synchronization behind it; the target block indicates the data block with the largest sequence number with a synchronization state of completed synchronization;

[0103] For example, in some specific implementations, based on the file transfer bitmap, if it is determined that the file is still not synchronized successfully, then prepare to send the data blocks out of order. Figure 3 ,The data status is divided into three states of ,synchronizing, waiting for synchronization and completing synchronization.,The sending strategy is to give priority to sending breakpoint data, ,followed by sending them one by one in sequence, and the strategy of ,the maximum fixed number of data to be synchronized is adopted.

[0104] Because the data is sent and received out of order and stored, and the data reception is confirmed based on the bitmap, when the data is lost, some of the bitmap may not be received. Figure 3, block 2, block 4 and block 6 belong to the type of packet loss retransmission. When a block is not received by the shore, it means that the network may be congested, so the block data is retransmitted here. The advantage of doing so is to try our best to ensure that all data is sent as soon as possible, thereby improving the transmission speed.

[0105] When sending data packets, the "breakpoint" situation will be counted first and sent first. The maximum number of packets sent is WindowSize1 (the window size can be customized). If the breakpoint data exceeds WindowSize1, the breakpoint data will be sent first. The sending of this breakpoint data is retransmission of resending data. This is done to facilitate regular data verification.

[0106] When the number of breakpoint data is less than or equal to , sequential data will be sent. At this time, fixed window moving sending is used. At most WindowSize2 data will be sent each time (the window size can be customized, at least 1, and the recommended range is 1-6). The data blocks in synchronization are at most WindowSize2. Whenever a transmission progress reply is received, based on the data reception on the shore side, the status of the data block is marked (some are changed from synchronizing to completed synchronization, and some are changed from waiting for synchronization to synchronizing), and then the sending window is moved and prepared to send new data blocks.

[0107] S300, regularly obtaining the block reception status of the shore end, and updating the full bitmap based on the block reception status;

[0108] It should be noted that, in some embodiments, periodically obtaining the block receiving status of the shore end may include at least one of the following steps:

[0109] When the number of data blocks received by the shore end in a round of data synchronization iteration is greater than the second number, obtaining the block reception status of the shore end in response to a reply from the shore end;

[0110] Block verification requests and heartbeat requests are sent to the shore end regularly, so that the shore end replies with the block reception status in response to the block verification request and / or heartbeat request, and then the block reception status of the shore end is obtained regularly.

[0111] For example, in some specific implementations, the shore end will receive a large number of data packets one after another and store them in sequence, but the shore end does not send a reply every time a data block is received, which will reduce the data transmission efficiency. The shore end will immediately reply to the data block reception status if the following three conditions are met:

[0112] Case 1: When receiving (If it cannot be divided evenly, round up to an integer) valid data blocks, the data block reception status will be replied immediately. The valid data blocks here do not include duplicate packets. For example, if WindowsSize2 is 3, the data reception order is data packet 1 (block 5), data packet 2 (block 5), data packet 3 (block 6), and data packet 4 (block 6). When data packet 3 is received, it means that 2 valid data blocks (block 5 and block 6) are received, and the data block reception status will be replied at this time. Data packets 2 and 4 are duplicate data packets and will be discarded. It can be seen from here that under normal circumstances, the number of data retransmissions on the ship side is mostly indivual.

[0113] Case 2: When receiving a block verification request, immediately reply with the data block reception status.

[0114] Case 3: When receiving a heartbeat request, immediately reply with the data block reception status.

[0115] In general, if any of Case 1, Case 2 and Case 3 is met, the data block reception status will be actively sent and replied, including file ID, bitmap start ID (also called start block ID), incremental bitmap information and other contents. Because the bitmap itself may be very large, what is transmitted here is not the entire bitmap, but the incremental bitmap information. The bitmap is basically transmitted according to the BitMapSize size (which can be customized and adjusted, generally 1K, i.e. 1024). Based on the bitmap start ID, it can be confirmed that the starting range of the bitmap is [Bitmap start ID, Bitmap start ID + BitMapSize]. This can be understood as the latest received BitMapSize block bitmap. The bitmap start ID is dynamically calculated, which means that the data packet of [0, start block ID) has been synchronized, and the current synchronization is the data packet of [Bitmap start ID, Bitmap start ID + BitMapSize). Incremental bitmaps can reduce the pressure of transmission bandwidth and increase the transmission speed.

[0116] It should also be noted that the block reception status includes the bitmap starting ID and incremental bitmap information, the bitmap starting ID corresponds to the largest sequence number of all data blocks that have completed data synchronization; the bitmap starting ID represents the sequence number corresponding to the starting bitmap, the synchronization status of the starting bitmap is completed synchronization, and the starting bitmap represents that there is no data block with the largest sequence number in the synchronization status before; in some embodiments, updating the full bitmap based on the block reception status may include the following steps: updating the synchronization status of the starting bitmap and all the data blocks before it to completed synchronization; wherein the incremental bitmap information includes the synchronization status of a preset number of data blocks behind the data block corresponding to the starting bitmap; based on the incremental bitmap information, updating the synchronization status of the preset number of data blocks behind the starting bitmap; wherein the status update includes updating the synchronization status of the data blocks to be transmitted in the next round of data synchronization iteration from waiting for synchronization to synchronizing.

[0117] In some embodiments, the method may further include starting and stopping data synchronization transmission according to the timeliness of the heartbeat request sent and the heartbeat reply of the shore end in response to the heartbeat request. The specific implementation may be as follows:

[0118] The ship will periodically send a heartbeat request, which includes information such as heartbeat SeqID and file ID. Each time a heartbeat is sent, SeqID = SeaID + 1. The sending frequency SendFrequency can be customized and is generally sent once every 1s (recommended between 500ms and 1s). The main reasons for setting a heartbeat request are as follows:

[0119] 1) Determine the network status. The ship can determine the network status through the heartbeat request reply. If the heartbeat reply is not received for many times, it means that the network quality is particularly poor. At this time, the sending progress will be adjusted or even stopped.

[0120] 2) Improve the transmission speed. The shore side will update the bitmap information immediately after receiving the heartbeat, and the ship side will send data based on the latest bitmap. Because the heartbeat is periodic, this mechanism can effectively avoid the extreme situation where the ship side can never obtain the latest bitmap due to the loss of other data packets, speed up the movement of the ship side's sending window, and improve the transmission speed.

[0121] After receiving the heartbeat request, the shore end will process it and reply to the ship end with the heartbeat SeqID, file ID, bitmap start ID, incremental bitmap information, etc. The ship end needs to make a network quality judgment based on the heartbeat reply. The present invention judges whether the data is lost based on the bitmap information, so there is no need to judge whether the data is lost.

[0122] Network quality determination: Since the heartbeat packet is very simple, the time consumed by the shore end to send and the ship end to receive and process is almost negligible. The network quality can be directly reflected by the round-trip of the heartbeat. If no heartbeat reply packet is received for a continuous period of M * SendFrequency (M can be customized, usually 3), that is, M heartbeat packets have no reply, it can be determined that the network quality is extremely poor at this time, and all data transmissions will be immediately stopped, but the sending frequency of the heartbeat packet remains unchanged.

[0123] If a heartbeat reply is received, assume the received heartbeat reply is SeqID1 and the latest sent heartbeat is SeqID2. If SeqID2 - SeqID1 >= M, it indicates that the network quality is extremely poor, resulting in a long round-trip time for the heartbeat. At this time, data transmission is still stopped. If SeqID2 - SeqID1 < M, it means the network quality is acceptable, and at this time, data packet transmission can be resumed or continued. Generally speaking, the heartbeat is a fallback method for the ship end to determine the network. It can not only improve the transmission speed but also perform breakpoint retransmission in a timely manner after the network returns to normal.

[0124] S400: Return to the step of performing breakpoint detection on the data chunks based on the full bitmap to obtain breakpoint data, and perform a new round of data synchronization iteration until all data chunks of the compressed package are synchronized.

[0125] Refer to Figure 4 , Figure 4 FIG. is the flowchart of the ship-shore data out-of-order synchronization method applied to the server provided by the embodiment of the present invention. The execution subject of the ship-shore data out-of-order synchronization method can be any of the aforementioned computer devices (including servers or terminals). Refer to Figure 4 ,this method is applied to the shore end and includes the following steps:

[0126] T100: In response to the data synchronization request from the ship end, obtain the compression information of the compression package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization situation; send the full bitmap to the ship end;

[0127] Among them, the full bitmap includes the sequence number and synchronization status of each data chunk of the compression package; the synchronization status includes completed synchronization, in-synchronization, and waiting for synchronization;

[0128] It should be noted that the specific implementation manner and application principle of this step correspond to those of the method in the first aspect described above, and will not be elaborated here.

[0129] T200: Receive the data chunks sent by the ship end in an out-of-order manner, and store the received data chunks in sequence based on the sequence number;

[0130] Exemplarily, in some specific implementations, the shore end will receive multiple UDP data packets, which include three parts of information: file ID, block ID, and data block. The file ID is used to determine which file the data belongs to, and the block ID is used to determine the block position. The block ID is the position in the bitmap. Taking into account network factors and breakpoint retransmission, the data packets received by the shore end may not arrive in order, but the shore end will store them in order according to the block ID. Those that have not been received temporarily belong to the "breakpoint" data. When the bitmap is synchronized, the ship end will retransmit according to the "breakpoint" data. At the same time, the shore end may receive data packets with the same file ID and block ID multiple times. If they have been saved, the duplicate data packets will be discarded directly. These packets may be retransmitted data packets.

[0131] refer to Figure 5 The data transmission process is normally repeated between the ship and the shore. The shore will periodically check the data and reply to the ship. When data loss is confirmed, the "breakpoint" data will be retransmitted first. When the "breakpoint" data is almost finished, new data will be sent again in sequence until all data is received successfully.

[0132] T300, based on the storage status of the data blocks, regularly reply the block reception status to the ship side, so that the ship side updates the full bitmap of the ship side based on the block reception status;

[0133] It should be noted that the ship side periodically sends block verification requests and / or heartbeat requests to the shore side. In some embodiments, periodically replying the block reception status to the ship side includes at least one of the following steps: when the number of data blocks received in a round of data synchronization iteration is greater than the second number, replying the block reception status to the ship side; in response to the block verification requests and / or heartbeat requests periodically sent by the ship side, replying the block reception status to the ship side.

[0134] It should be noted that the specific implementation method of this step and its application principle correspond to the principle of the method of the first aspect mentioned above and will not be repeated here.

[0135] In some embodiments, the ship sends a block verification request to the shore periodically, such as Figure 6 As shown, the method may also include the following steps: P100, in response to a block check request from the ship side, obtaining a first check value of a data block sent by the ship side; P200, obtaining a second check value of the received data block by processing with a preset information summary algorithm; P300, performing a data consistency check based on the first check value and the second check value corresponding to each synchronized data block; P400, when the data consistency check fails, sending a retransmission request for the corresponding data block to the ship side, so that the ship side retransmits the corresponding data block in response to the retransmission request.

[0136] For example, in some specific implementations, when a verification request is received, the data is fetched for MD5 verification according to the start block verification ID and the end block verification ID, and an MD5 verification value is obtained. Assume that the MD5 verification value calculated by the shore end is A1, and the MD5 verification value calculated by the ship end is A2.

[0137] If A1 and A2 are consistent, the reply is verified consistent and the latest incremental bitmap is replied.

[0138] If A1 and A2 are inconsistent, it means that the data may be tampered with, then the data of [starting block verification ID, ending block verification ID) is cleared, the corresponding bitmap state is reset to wait for synchronization, and the latest bitmap on the ship side is updated (that is, the latest starting block ID is set as the starting block verification ID), and then the verification is inconsistent and the latest incremental bitmap is replied. This is also one of the core points of the present invention, that is, the data that failed the block verification can be actively retransmitted.

[0139] If the verification is consistent, update the latest start verification block ID to the end verification block ID, and update the latest bitmap at the same time.

[0140] If the checksum is inconsistent, the data of [starting block checksum ID, ending block checksum ID) is cleared, and the bitmap on the ship side is directly overwritten based on the latest bitmap, preparing to enter the "breakpoint" data retransmission stage. Based on experience, this situation is rare.

[0141] It is important to note here that only when the received data verification is inconsistent will the shore-side incremental bitmap be used to directly overwrite the ship-side bitmap information. In other cases, the bitmap is synchronized based on the largest and latest starting block ID.

[0142] In some embodiments, the method may further include the following steps: after all data blocks are synchronized, an overall verification of the synchronized data is performed. This may be implemented as follows:

[0143] Based on the ship-side bitmap, the ship-side determines that all data has been received, indicating that data synchronization is successful. At this time, a verification request will be sent, and the request content is mainly the information of the current transmission file (file name, file ID, MD5, file size, last modification time, etc.).

[0144] After receiving the verification request, the shore end will recalculate the MD5 of the file and then compare it with the requested MD5. If they are consistent, it means that the file transfer is successful. If they are inconsistent, it means that some data packets may have been tampered with. In most cases, the MD5 is consistent.

[0145] T400, return to the step of receiving the data blocks sent by the ship in disorder, and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

[0146] In order to explain the principle of the technical solution of the present invention in detail, the overall process of the present invention is described below in combination with some specific embodiments. It is easy to understand that the following is an explanation of the technical principle of the present invention and cannot be regarded as a limitation of the present invention.

[0147] First of all, it should be explained that the current mainstream solution for ship-shore data synchronization is: Currently, data synchronization from the ship end to the shore end is mainly based on TCP, which is consistent with the idea of ​​data synchronization between two points on land, that is, a TCP communication channel is established between the two points, and then the synchronization data is sent.

[0148] Existing work ideas: Currently, data synchronization from the ship to the shore is divided into three stages: metadata preparation, data synchronization and final verification. The transmission characteristics are sequential segmented transmission, and sequential breakpoint resumption, which makes full use of idle bandwidth and improves transmission speed.

[0149] However, the current mainstream solutions have the following disadvantages:

[0150] 1) Slow speed: Currently, data from ship to shore is mainly transmitted via satellite. The satellite bandwidth is relatively low, resulting in slow data synchronization.

[0151] 2) Poor experience: Considering the low satellite bandwidth and the fact that data synchronization and onboard technicians need to be connected to the Internet, there will be bandwidth competition, resulting in a poor Internet experience for users.

[0152] 3) Low synchronization success rate: Currently, data synchronization is mainly based on TCP transmission. The integrity of data transmission is theoretically guaranteed, but considering the low bandwidth of satellites, data transmission often fails halfway through.

[0153] Disadvantages of existing work plans:

[0154] 1) More retransmission bandwidth: Because existing work is sent and retransmitted sequentially, a lot of received data will still be retransmitted, resulting in a large amount of retransmitted data and the transmission speed still needs to be improved.

[0155] 2) Low synchronization success rate: It is easy for data synchronization to exceed 80%, but due to network reasons, it causes continuous retransmission, which eventually leads to data synchronization failure.

[0156] Data is easily tampered with: With only a final verification mechanism, intermediate data is prone to tampering, resulting in failure of final data synchronization.

[0157] In view of this, the present invention proposes a method for synchronizing the data of the ship-to-shore number out of order, referring to Figure 7 , the specific steps are as follows:

[0158] Step 1: Local data compression. There are usually many files to be transmitted on the ship side, mainly the test results of technical experiments, in formats including tables, texts, pictures and other files, and fewer high-definition videos. These data files need to be compressed before transmission. This article uses the open source LZMA2 compression algorithm, which has a very high compression rate. Compared with other compression algorithms, the compression speed will be slightly slower, but considering the satellite bandwidth, a higher compression rate is still necessary. In the data transmission management regulations on board, it is not recommended to transmit high-definition video. This is because current videos are usually in compressed format. Even if LZMA2 is used for re-compression, the file compression rate will not be too high, and the video will occupy a lot of bandwidth and affect the transmission of other files. Therefore, high-definition video files are usually synchronized by direct copying after the ship docks.

[0159] Under normal circumstances, there will be multiple files to be synchronized. These files are sorted according to the last modification time, and the files will be divided into multiple groups for compression. This article uses 100M as the benchmark for grouping, that is, the sum of the file sizes of each group is around 100M. If a single file exceeds 100M, then this file is grouped separately. Then compression is performed for each group, and the compressed package name is the MD5 of all file names added together, so that a series of compressed packages will be obtained. This grouping standard can be customized, but it is not recommended to exceed 200M, because the larger the compressed file, the higher the file transfer failure rate. These compressed packages are transmitted synchronously one by one, and the present invention is aimed at single compressed packages for data synchronization.

[0160] In most cases, the compression rate of text files can reach about 60%, that is, the size of the compressed text file is about 40% of the original size. According to the 100M grouping, a single compressed package is about 40M. In actual scenarios, the compression rate can even reach 70%.

[0161] Regarding grouping, for example, if 7 files A (size 80M), B (size 17M), C (size 60M), D (size 60M), E (size 70M), F (size 80M), G (170M) have been sorted by the last modification time, then the grouping is as follows:

[0162] 1) A (size 80M) and B (size 17M) are a group, because A+B=97M, which is around 100M, the compressed package name is MD5(AB);

[0163] 2) C (size 60M) and D (size 60M) are a group. C is 60M, which is 40M away from 100M, but C+D=120M, which is only 20M more than 100M. Therefore, C and D are a group, and the compressed package name is MD5(CD);

[0164] 3) E (size 70M) is a separate group. E is 70M, 30M away from 100M, but E+F=150M, 50M more than 100M, so E is a separate group, and the compressed package name is MD5(E);

[0165] 4) F (size 80M) is a separate group for the same reason as E;

[0166] 5) G (size 170M) is a separate group;

[0167] To sum up, it is divided into 5 groups, each of which is compressed into a package using LZMA2.

[0168] Step 2: Initiate a ship-side data synchronization request. The ship sends a data synchronization request, which is forwarded to the shore via satellite. It mainly includes detailed information about the compressed package of transmitted data, including the file name, file size, file MD5, file creation time, last modification time, etc. This information is mainly used to determine whether breakpoint resuming is required.

[0169] Step 3: Check data synchronization. When the shore side receives the data synchronization request, it will check the files that have been transferred on the shore side. The judgment process is mainly as follows:

[0170] 1) Check the file name to see if the file exists on the shore. If it does not exist, it means that the file is being transmitted for the first time. If it does not exist, continue to compare the file's MD5;

[0171] 2) Check the file MD5 to see if the MD5 of the file on the shore is consistent with the MD5 of the target file. If they are inconsistent, it means that the target file has changed and needs to be retransmitted. If they are consistent, continue to compare the file sizes.

[0172] 3) Check the bitmap information and file size. If they are consistent, it means the file has been transferred before and you can choose to resume the transfer. If they are inconsistent, it means the target file has changed and needs to be transferred again. This step is necessary because the file is stored in the cache before it is successfully transferred, and the cache may be tampered with or cleared.

[0173] 4) Check the bitmap information. If all data packets have been received, it means that the data synchronization is successful. Otherwise, it needs to be retransmitted or retransmitted from the breakpoint. If the data has been transmitted successfully, check the final verification mark on the shore side. If the verification has been completed, reply that the ship side synchronization is successful. Otherwise, reply that the final verification is required.

[0174] The bitmap information is used to mark the situation of the shore receiving the file. Assuming that the compressed file is 50M, each data packet is transmitted using the UDP protocol, and each packet is about 1K, then there are a total of 50M / 1K=51200 bits, that is, 51200 bits are needed to represent the status of each packet, 0 means not received, 1 means received, that is, the bitmap size is 51200 bits, and 51200 / 8=6.25K bytes are needed to represent the bitmap. The bitmap is mainly used for data retransmission, reducing duplicate bandwidth and increasing transmission speed.

[0175] Step 4: Data synchronization status reply on the shore side. After the inspection is completed, the monitoring status will be directly fed back to the ship side, mainly the file information that already exists on the shore side (file name, MD5, file bitmap, file size, last modified time, transmission progress, etc.). Note that the bitmap here is the full bitmap, that is, the entire bitmap needs to be synchronized to the ship side.

[0176] Step 5: Send data blocks out of order. Based on the file transfer bitmap, if it is determined that all data packets have been received, a final check of the data is required. The ship side checks whether the final check of the file has been completed. If the file has completed the final check, the entire synchronous transmission process ends. If the file has not been finalized, go to step 13 to perform data verification.

[0177] Based on the file transfer bitmap, if it is determined that the file has not been synchronized successfully, prepare to send data blocks out of order. Figure 3 ,The data status is divided into three states of ,synchronizing, waiting for synchronization and completing synchronization.,The sending strategy is to give priority to sending breakpoint data, ,followed by sending them one by one in sequence, and the strategy of ,the maximum fixed number of data to be synchronized is adopted.

[0178] Because the data is sent and received out of order and stored, and the data reception is confirmed based on the bitmap, when the data is lost, some of the bitmap may not be received. Figure 3 , blocks 2, 4 and 6 belong to the type of packet loss retransmission. When a block is not received by the shore, it means that the network may be congested, so the block data is retransmitted here. The advantage of doing so is to try to ensure that all data is sent as soon as possible, thereby improving the transmission speed. As for how to judge whether the shore has received the data block, the judgment rules will be given in step 12 later, which will not be expanded here.

[0179] When sending data packets, the "breakpoint" situation will be counted first and sent first. The maximum number of packets sent is WindowSize1 (the window size can be customized). If the breakpoint data exceeds WindowSize1, the breakpoint data will be sent first. The sending of this breakpoint data is retransmission of resending data. This is done to facilitate regular data verification.

[0180] When the number of breakpoint data is less than or equal to , sequential data will be sent. At this time, fixed window moving sending is used. At most WindowSize2 data will be sent each time (the window size can be customized, at least 1, and the recommended range is 1-6). The data blocks in synchronization are at most WindowSize2. Whenever a transmission progress reply is received, based on the data reception on the shore side, the status of the data block is marked (some are changed from synchronizing to completed synchronization, and some are changed from waiting for synchronization to synchronizing), and then the sending window is moved and prepared to send new data blocks.

[0181] For example, refer to Figure 2 , assuming that WindowSize1 is 2 and WindowSize2 is 3, then blocks 2, 4 and 6 belong to the type of packet loss retransmission, and need to be retransmitted first. When blocks 2 and 4 are synchronized, only block 6 is left for packet loss retransmission, that is, the breakpoint data is equal to 1. At this time, the sequential transmission starts from block 9, that is, blocks 9, 10 and 11 are sent. The priority retransmission of lost packet data ensures that there is no "breakpoint" in the received data as much as possible, laying a solid foundation for the regular segmentation check of the subsequent data.

[0182] Step 6: Receive data blocks out of order and store them in order. The shore side will receive multiple UDP data packets, which include three parts of information: file ID, block ID, and data block. Determine which file the data belongs to based on the file ID, and determine the block position based on the block ID. The block ID is the position in the bitmap. Taking into account network factors and breakpoint retransmission, the data packets received by the shore side may not arrive in order, but the shore side will store them in order according to the block ID. Those that have not been received temporarily belong to "breakpoint" data. When the bitmap is synchronized, the ship side will retransmit according to the "breakpoint" data. At the same time, the shore side may receive data packets with the same file ID and block ID multiple times. If they have been saved, the duplicate data packets will be discarded directly. These packets may be retransmitted data packets.

[0183] The bitmap is updated each time a packet is received and is synchronized periodically according to the rules described in step 8.

[0184] refer to Figure 5 The data transmission process is normally repeated between the ship and the shore. The shore will periodically check the data and reply to the ship. When data loss is confirmed, the "breakpoint" data will be retransmitted first. When the "breakpoint" data is almost finished, new data will be sent again in sequence until all data is received successfully.

[0185] Step 7: Regularly reply to the data block reception status. The shore end will receive a large number of data packets one after another and store them in sequence, but the shore end does not send a reply every time it receives a data block, which will reduce the data transmission efficiency. The shore end will immediately reply to the data block reception status if the following three conditions are met:

[0186] Case 1: When receiving (If it cannot be divided evenly, round up to an integer) valid data blocks, the data block reception status will be replied immediately. The valid data blocks here do not include duplicate packets. For example, if WindowsSize2 is 3, the data reception order is data packet 1 (block 5), data packet 2 (block 5), data packet 3 (block 6), and data packet 4 (block 6). When data packet 3 is received, it means that 2 valid data blocks (block 5 and block 6) are received, and the data block reception status will be replied at this time. Data packets 2 and 4 are duplicate data packets and will be discarded. It can be seen from here that under normal circumstances, the number of data retransmissions on the ship side is mostly indivual.

[0187] Case 2: When receiving a block verification request, immediately reply with the data block reception status.

[0188] Case 3: When receiving a heartbeat request, immediately reply with the data block reception status.

[0189] In general, if any of Case 1, Case 2 and Case 3 is met, the data block reception status will be actively sent and replied, including file ID, bitmap start ID (also called start block ID), incremental bitmap information and other contents. Because the bitmap itself may be very large, what is transmitted here is not the entire bitmap, but the incremental bitmap information. The bitmap is basically transmitted according to the BitMapSize size (which can be customized and adjusted, generally 1K, i.e. 1024). Based on the bitmap start ID, it can be confirmed that the starting range of the bitmap is [Bitmap start ID, Bitmap start ID + BitMapSize]. This can be understood as the latest received BitMapSize block bitmap. The bitmap start ID is dynamically calculated, which means that the data packet of [0, start block ID) has been synchronized, and the current synchronization is the data packet of [Bitmap start ID, Bitmap start ID + BitMapSize). Incremental bitmaps can reduce the pressure of transmission bandwidth and increase the transmission speed.

[0190] Here is an example, assuming that the compressed file is 50M, each data packet is transmitted using the UDP protocol, and each packet is about 1K, then there are a total of 50M / 1K=50000 bits, that is, 50000 bits are needed to represent the status of each packet, 0 means not received, 1 means received, that is, the bitmap size is 50000 / 8=6250=6.1K bytes. The 6.1K byte bitmap is still relatively large, so here we choose to transmit the incremental bitmap, assuming the starting ID is 0, BitMapSize is 1024, then the bitmap received in blocks is [0,1024).

[0191] Regarding the "breakpoint" retransmission strategy in step 5, there is one thing that needs to be discussed here. The shore side will send the data block reception status according to the fixed rules of Case 1, Case 2 and Case 3. When the ship side receives the latest bitmap information, it will actively check the "breakpoint" data status, stop the current data transmission, and give priority to sending the "breakpoint" data. When multiple incremental bitmap information is received, the largest starting bitmap ID is used as the latest progress (except for the special case of data verification failure, in which case the ship side bitmap will be directly overwritten and enter the state of large breakpoint retransmission).

[0192] Step 8: Send block verification requests periodically. When the ship receives the incremental bitmap, it will synchronize the bitmap and mainly complete the following operations.

[0193] 1. Check the data block of synchronization [0, starting block ID), set the status to synchronization completed, synchronize and update the bitmap on the ship side.

[0194] 2. Based on the incremental bitmap [starting block ID, bitmap starting ID + 1K), directly overwrite, synchronize and update the bitmap on the ship side.

[0195] At the same time, the ship side may receive incremental bitmap information multiple times. At this time, the largest starting block ID is used as the latest bitmap, and other incremental bitmap information will be discarded. After the ship side updates to the latest bitmap, the ship side will periodically issue a data verification request. The ship side will issue a verification request based on the latest starting verification block ID and the data verification size VerifSize (which can be customized and adjusted, the default is 100). Assume that the latest starting block ID is LastRecvID and the latest starting verification block ID is LastVerifID.

[0196] Calculate BatchSize according to the following formula

[0197]

[0198] If BatchSize is 0, it means that the latest received data is less than VerifSize, and no verification request is issued. If BatchSize>0, the shore will calculate the MD5 checksum of [LastVerifID,LastVerifID+BatchSize*VerifSize), and then issue a data verification request in the range of [LastVerifID,LastVerifID+BatchSize*VerifSize).

[0199] If LastRecvID is the last ID of the bitmap, that is, all data synchronization is completed, a data verification request in the range of [LastVerifID, LastRecvID] is issued.

[0200] The data verification request package content includes file ID, start block verification ID, end block verification ID, and MD5 verification value.

[0201] For example, if LastRecvID is 1027, LastVerifID is 200, VerifSize is 100, and batchSize is 8, then a block ID check in the range of [200, 1000) is issued.

[0202] Here we explain how to set the size of VerifSize. The current bandwidth of ship-shore students is about 2.5MB, that is, 2500KB of bandwidth. Considering other bandwidth usage on the ship, it is assumed that the data synchronization bandwidth is 1KB. VerifSize takes 100 (each package is 1K, 100 is 100s), which means that data verification is performed every 100s. VerifSize is set according to the current satellite bandwidth. It should not be too large. Based on the current satellite bandwidth, it is recommended not to exceed 500.

[0203] Step 9: Verify the received data. When receiving the verification request, the data will be fetched for MD5 verification based on the start block verification ID and the end block verification ID, and the MD5 verification value will be obtained. Assume that the MD5 verification value calculated by the shore end is A1, and the MD5 verification value calculated by the ship end is A2.

[0204] If A1 and A2 are consistent, the reply is verified to be consistent and the latest incremental bitmap is replied.

[0205] If A1 and A2 are inconsistent, it means that the data may be tampered with, then the data of [starting block verification ID, ending block verification ID) is cleared, the corresponding bitmap state is reset to wait for synchronization, and the latest bitmap on the ship side is updated (that is, the latest starting block ID is set as the starting block verification ID), and then the verification is inconsistent and the latest incremental bitmap is replied. This is also one of the core points of the present invention, that is, the data that failed the block verification can be actively retransmitted.

[0206] Step 10: Respond to the data verification status and selectively retransmit.

[0207] If the verification is consistent, update the latest start verification block ID to the end verification block ID, and update the latest bitmap at the same time.

[0208] If the checksum is inconsistent, the data of [starting block checksum ID, ending block checksum ID) is cleared, and the bitmap on the ship side is directly overwritten based on the latest bitmap, preparing to enter the "breakpoint" data retransmission stage. Based on experience, this situation is rare.

[0209] It is important to note here that only when the received data verification is inconsistent will the shore-side incremental bitmap be used to directly overwrite the ship-side bitmap information. In other cases, the bitmap is synchronized based on the largest and latest starting block ID.

[0210] Step 11: Send heartbeat request. The ship will periodically send heartbeat request, the request content includes heartbeat SeqID, file ID and other information, each time a heartbeat is sent, SeqID = SeaID + 1. The sending frequency SendFrequency can be customized, generally once every 1s (recommended between 500ms and 1s). The main reasons for setting the heartbeat request are as follows:

[0211] 1) Determine the network status. The ship can determine the network status through the heartbeat request reply. If the heartbeat reply is not received for many times, it means that the network quality is particularly poor. At this time, the sending progress will be adjusted or even stopped.

[0212] 2) Improve the transmission speed. The shore side will update the bitmap information immediately after receiving the heartbeat, and the ship side will send data based on the latest bitmap. Because the heartbeat is periodic, this mechanism can effectively avoid the extreme situation where the ship side can never obtain the latest bitmap due to the loss of other data packets, speed up the movement of the ship side's sending window, and improve the transmission speed.

[0213] Step 12: Heartbeat reply. After receiving the heartbeat request, the shore end will process it and reply the heartbeat SeqID, file ID, bitmap start ID, incremental bitmap information and other information to the ship end. The ship end needs to make a network quality judgment based on the heartbeat reply. The present invention judges whether the data is lost based on the bitmap information, so there is no need to judge whether the data is lost.

[0214] Network quality determination: Since the heartbeat packet is very simple, the time consumed by the shore end to send and the ship end to receive and process is almost negligible. The network quality can be directly reflected by the round-trip of the heartbeat. If no heartbeat reply packet is received for a continuous time of M * SendFrequency (M can be customized, usually 3), that is, M heartbeat packets have no reply, it can be determined that the network quality is extremely poor at this time, and all data transmissions will be immediately stopped, but the sending frequency of the heartbeat packet remains unchanged.

[0215] If a heartbeat reply is received, assume the received heartbeat reply is SeqID1 and the latest sent heartbeat is SeqID2. If SeqID2 - SeqID1 >= M, it means the network quality is extremely poor, resulting in a very long heartbeat round-trip time. At this time, data transmission is still stopped. If SeqID2 - SeqID1 < M, it means the network quality is acceptable, and at this time, data packet transmission can be resumed or continued. Generally speaking, the heartbeat is a fallback method for the ship end to determine the network. It can not only improve the transmission speed but also enable resume data transmission from the breakpoint in a timely manner after the network returns to normal.

[0216] Here is an example. Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, receive heartbeat reply 1, send heartbeat 2, receive heartbeat reply 2, send heartbeat 3, receive heartbeat reply 3. In this case of normal data network, the general network delay on land is generally 30ms to 50ms. Considering satellite transmission, the ship-shore network delay is estimated to be 200ms, that is, the reply of heartbeat 1 can be received after 200ms. So the above belongs to the normal network situation.

[0217] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, send heartbeat 2, receive heartbeat reply 1, receive heartbeat reply 2, send heartbeat 3, receive heartbeat reply 3. This situation belongs to a slightly worse network, but data transmission can still continue.

[0218] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 1, send heartbeat 2, send heartbeat 3, send heartbeat 4. At this time, it belongs to the situation where no reply is received for M consecutive heartbeats, indicating that the network quality is very poor, and all data transmissions will be immediately stopped.

[0219] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 4, receive heartbeat reply 1. At this time, although a heartbeat reply is received, this reply is from a long time ago, and all data transmissions will still be stopped.

[0220] Assume the ship end's heartbeat sending and receiving situation is as follows: send heartbeat 4, receive heartbeat reply 1, receive heartbeat reply 4. At this time, it indicates that the network situation has improved, and data transmission will be resumed.

[0221] Step 13: Data synchronization is successful, and a verification request is sent. Based on the ship-side bitmap, the ship-side determines that all data has been received, indicating that data synchronization is successful. At this time, a verification request will be sent, and the request content is mainly the information of the current transmission file (file name, file ID, MD5, file size, last modification time, etc.).

[0222] Step 14: Local data verification. After receiving the verification request, the shore end will recalculate the MD5 of the file and then compare it with the requested MD5. If they are consistent, it means that the file transfer is successful. If they are inconsistent, it means that some data packets may have been tampered with. In most cases, the MD5 is consistent.

[0223] Step 15: Reply with data verification status. The shore side replies with the MD5 verification status, which is mainly the information of the currently transmitted file (file name, file ID, MD5, file size, last modification time, etc.). After receiving the reply, the shore side records the status of this transmission, marks that the file has been synchronized, and notifies the user. If the MD5 is consistent, the user is notified that the file transfer is successful, otherwise the user is notified that the intermediate data has been tampered with and needs to be resynchronized (the data will not be reset and cleared here, and the user needs to actively retransmit). Simply put, data that fails the block verification can be actively retransmitted, but if it reaches the final verification step, the data that fails the verification will not be actively retransmitted. The user will be reminded and can choose to retransmit.

[0224] When steps 13 to 15 are completed, the file will be marked as synchronized, and then another file will be transferred. There is an extreme case that needs to be explained. If the file has just been successfully transferred, and the network is suddenly disconnected, the ship side cannot receive the data verification reply for a long time. At this time, the file is still in the synchronization state. When the network is restored to normal, the ship side will resend the verification request until the verification reply is received. The transmission process of the file is truly completed.

[0225] To sum up, the entire transmission is divided into three stages:

[0226] 1) Preparation phase, including steps 1 to 4. This phase mainly clarifies the information of the file to be transmitted on the shore side and synchronizes the latest bitmap. It is the data preparation phase before transmission.

[0227] 2) The synchronization phase includes steps 5 to 12. This phase is mainly responsible for data synchronization and bitmap synchronization. The sending rhythm is dynamically adjusted through the heartbeat and block verification mechanism to improve file transfer efficiency.

[0228] 3) Verification phase, including steps 13 to 15, this phase is mainly the final data verification.

[0229] Compared with the prior art, the present invention has at least the following beneficial effects:

[0230] 1) Introduce a first-level bitmap mechanism to control the data sending process, reduce data retransmission bandwidth, and increase transmission speed.

[0231] 2) Based on the fixed window sending strategy, out-of-order sending is realized to make full use of idle bandwidth and improve transmission speed.

[0232] 3) Intermediate data is verified regularly and supports breakpoint resumption, ensuring data integrity from the start to the completion of data download.

[0233] On the other hand, Figure 8 As shown, an embodiment of the present invention provides a ship-shore data out-of-order synchronization device 800, which is applied to a ship side and may include:

[0234] The first module 801 is used to initiate a data synchronization request to the shore end, and send the compression information of the compressed package to be synchronized to the shore end, so that the shore end performs data synchronization detection based on the compression information and obtains a full bitmap of the data synchronization status of the shore end;

[0235] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0236] The second module 802 is used to perform breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the amount of breakpoint data is greater than the first amount, resend the breakpoint data to the shore end; otherwise, synchronize the data block sequence after the target block to the shore end;

[0237] The synchronization state of the breakpoint data is in sync, and the breakpoint data indicates that there is a data block with a synchronization state of completed synchronization behind it; the target block indicates the data block with the largest sequence number with a synchronization state of completed synchronization;

[0238] The third module 803 is used to periodically obtain the block reception status of the shore end, and update the full amount bitmap based on the block reception status;

[0239] The fourth module 804 is used to return to execute the second module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0240] On the other hand, Fig. 9 As shown, an embodiment of the present invention provides a ship-shore data out-of-order synchronization device 900, which is applied to the shore side and may include:

[0241] The fifth module 901 is used to respond to the data synchronization request of the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization status; and send the full bitmap to the ship end;

[0242] The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization;

[0243] The sixth module 902 is used to receive the data blocks sent by the ship side out of order, and store the received data blocks in order based on the sequence number;

[0244] The seventh module 903 is used to periodically reply the block reception status to the ship end based on the storage status of the data blocks, so that the ship end updates the full bitmap of the ship end based on the block reception status;

[0245] The eighth module 904 is used to return to execute the sixth module and perform a new round of data synchronization iteration until all data blocks of the compressed package are synchronized.

[0246] In some embodiments, the ship side periodically sends a block verification request to the shore side, and the ship-shore data out-of-order synchronization device applied to the shore side may also include:

[0247] A ninth module is configured to obtain, in response to a block check request from the ship side, a first check value of a data block sent by the ship side;

[0248] A tenth module is used to obtain a second check value of the received data block by processing through a preset information summary algorithm;

[0249] An eleventh module is used to perform data consistency verification based on a first verification value and a second verification value corresponding to each synchronized data block;

[0250] The twelfth module is used to send a retransmission request of the corresponding data block to the ship side when the data consistency check fails, so that the ship side retransmits the corresponding data block in response to the retransmission request.

[0251] The contents of the method embodiments of the present invention are all applicable to the device embodiments. The functions specifically implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0252] On the other hand, an embodiment of the present invention further provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned hydrate stability region bottom boundary prediction method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0253] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0254] like Fig.10 As shown, Fig.10 The hardware structure of an electronic device 1000 of another embodiment is illustrated. The electronic device 1000 includes:

[0255] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0256] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the network node population optimization method of the embodiment of the present invention;

[0257] Input / output interface 1003, used to implement information input and output;

[0258] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0259] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0260] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0261] The electronic device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0262] The contents of the method embodiments of the present invention are all applicable to the electronic device embodiments. The functions specifically implemented by the electronic device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0263] Another aspect of an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the above method.

[0264] It should be noted that the computer-readable medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read to Only Memory, CD to ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0265] The contents of the method embodiments of the present invention are all applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0266] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0267] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0268] It should be noted that, although several modules of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0269] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD to ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present invention.

[0270] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented by the present invention. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part described as a larger operation is performed independently.

[0271] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of various functional modules in the device disclosed in the present invention, the actual implementation of the module will be understood within the conventional skills of engineers. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0272] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0273] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution device, apparatus or device (such as a computer-based device, a device including a processor, or other device that can fetch instructions from an instruction execution device, apparatus or device and execute the instructions), or in conjunction with such instruction execution device, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution device, apparatus or device, or in conjunction with such instruction execution device, apparatus or device.

[0274] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0275] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution device. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0276] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0277] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0278] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for out-of-order synchronization of ship-shore data, characterized in that: Applied to the ship side, the method comprises the following steps: A data synchronization request is initiated to the shore end, and the compression information of the compressed package to be synchronized is sent to the shore end, so that the shore end performs data synchronization detection based on the compression information and obtains a full bitmap of the data synchronization status of the shore end; The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization; Performing breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data; when the amount of the breakpoint data is greater than a first amount, resending the breakpoint data to the shore end; otherwise, synchronizing the data blocks after the target block to the shore end in sequence; The synchronization state of the breakpoint data is in synchronization, and the breakpoint data indicates that there is a data block whose synchronization state is completed synchronization; the target block indicates the data block with the largest sequence number whose synchronization state is completed synchronization; Periodically obtaining the block reception status of the shore end, and updating the full bitmap based on the block reception status; Return to the step of performing breakpoint detection on the data blocks based on the full bitmap to obtain breakpoint data, and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

2. The method for out-of-order synchronization of ship-shore data according to claim 1, characterized in that: The block receiving status of the shore end is obtained regularly. Includes at least one of the following steps: When the number of the data blocks received by the shore end in a round of data synchronization iteration is greater than a second number, obtaining the block reception status of the shore end in response to a reply from the shore end; Periodically send block check requests and heartbeat requests to the shore end, so that the shore end replies with the block reception status in response to the block check request and / or the heartbeat request, and then periodically obtain the block reception status of the shore end.

3. The method for out-of-order synchronization of ship-shore data according to claim 1, characterized in that: The block reception situation includes a bitmap start ID and incremental bitmap information, the bitmap start ID corresponds to the largest sequence number of all the data blocks that have completed data synchronization; the bitmap start ID represents the sequence number corresponding to the start bitmap, the synchronization state of the start bitmap is the completed synchronization, and the start bitmap represents that there is no data block with the largest sequence number in the synchronization state before; the updating of the full bitmap based on the block reception situation includes the following steps: Updating the synchronization status of the starting bitmap and all the data blocks preceding it to the synchronization completion; Wherein, the incremental bitmap information includes the synchronization status of a preset number of data blocks following the data block corresponding to the start bitmap; Based on the incremental bitmap information, updating the synchronization status of the preset number of data blocks after the starting bitmap; The state update includes updating the synchronization state of the data blocks to be transmitted in the next round of data synchronization iteration from waiting for synchronization to synchronizing.

4. A ship-shore data out-of-order synchronization device, characterized in that: Applied to ship side, including: The first module is used to initiate a data synchronization request to the shore end, and send the compressed information of the compressed package to be synchronized to the shore end, so that the shore end performs data synchronization detection based on the compressed information, and obtains a full bitmap of the data synchronization status of the shore end; The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization; The second module is used to perform breakpoint detection on the data block based on the full bitmap to obtain breakpoint data; when the amount of the breakpoint data is greater than the first amount, resend the breakpoint data to the shore end; otherwise, synchronize the data block sequence after the target block to the shore end; The synchronization state of the breakpoint data is in synchronization, and the breakpoint data indicates that there is a data block whose synchronization state is completed synchronization; the target block indicates the data block with the largest sequence number whose synchronization state is completed synchronization; The third module is used to periodically obtain the block reception status of the shore end, and update the full amount bitmap based on the block reception status; The fourth module is used to return to execute the second module and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

5. A method for out-of-order synchronization of ship-shore data, characterized in that: Applied to the shore side, the method is interactively applied with the ship-shore data out-of-order synchronization method according to any one of claims 1 to 3, and the method comprises the following steps: In response to a data synchronization request from the ship side, obtaining compression information of a compression package to be synchronized; performing data synchronization detection based on the compression information to obtain a full bitmap of data synchronization status; and sending the full bitmap to the ship side; The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization; receiving the data blocks sent by the ship end out of order, and sequentially storing the received data blocks based on the sequence numbers; Based on the storage status of the data blocks, periodically replying the block reception status to the ship end, so that the ship end updates the full quantity bitmap of the ship end based on the block reception status; Return to the step of receiving the data blocks sent by the ship in disorder, and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

6. The method for out-of-order synchronization of ship-shore data according to claim 5, characterized in that: The ship end periodically sends a block verification request and / or a heartbeat request to the shore end, and the ship end periodically replies to the ship end about the block reception status, Includes at least one of the following steps: When the number of the data blocks received in a round of data synchronization iteration is greater than a second number, replying the block reception status to the ship end; In response to the block check request and / or the heartbeat request periodically sent by the ship end, the block reception status is replied to the ship end.

7. The method for out-of-order synchronization of ship-shore data according to claim 5, characterized in that: The ship end periodically sends a block verification request to the shore end, and the method further comprises the following steps: In response to the block check request of the ship end, obtaining, from the ship end, a first check value of the data block sent by the ship end; Obtaining a second check value of the received data block by processing with a preset information digest algorithm; Performing data consistency verification based on the first verification value and the second verification value corresponding to each of the synchronized data blocks; When the data consistency check fails, a retransmission request of the corresponding data block is sent to the ship end, so that the ship end retransmits the corresponding data block in response to the retransmission request.

8. A ship-shore data out-of-order synchronization device, characterized in that: Applied to the shore side, the device is interactively applied with the ship-shore data out-of-order synchronization device as claimed in claim 4, comprising: The fifth module is used to respond to the data synchronization request of the ship end, obtain the compression information of the compressed package to be synchronized; perform data synchronization detection based on the compression information to obtain a full bitmap of the data synchronization situation; and send the full bitmap to the ship end; The full bitmap includes the sequence number and synchronization status of each data block of the compressed package; the synchronization status includes completed synchronization, synchronizing, and waiting for synchronization; A sixth module is used to receive the data blocks sent by the ship end out of order, and sequentially store the received data blocks based on the sequence numbers; A seventh module is used to periodically reply the block reception status to the ship end based on the storage status of the data blocks, so that the ship end updates the full amount bitmap of the ship end based on the block reception status; The eighth module is used to return to execute the sixth module and perform a new round of data synchronization iteration until all the data blocks of the compressed package are synchronized.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 3 or 5 to 7.

10. A computer storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to implement the method according to any one of claims 1 to 3 or 5 to 7 when executed by the processor.

Images