A method and system for real-time message storage of a marine data acquisition device

By merging and parsing ship communication data, converting it into PCAP format and storing it, the storage problem of ship data acquisition devices in high-traffic environments was solved, achieving stable and efficient data storage and analysis.

CN123123484APending Publication Date: 2026-11-17CHINA SHIP DEV & DESIGN CENT
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510984684.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

When the real-time communication data traffic of a ship is too large, the real-time message data storage of the ship's data acquisition device is incomplete, causing the device to freeze or crash, affecting fault analysis and anomaly monitoring.

Method used

Multiple individual messages are merged into an aggregate message using a programmable logic device, and then stored in a data receiving buffer queue by a data processor. After being parsed into a PCAP format message, it is stored using flexible configuration rules.

Benefits of technology

It enables effective storage of ship communication data under high traffic conditions, improves system efficiency and stability, avoids data acquisition device lag and crashes, and supports rapid fault analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123484A_ABST
    Figure CN123123484A_ABST
Patent Text Reader

Abstract

This invention discloses a real-time message storage method and system for a marine data acquisition device. The method includes: using a programmable logic device to merge multiple individual messages received by the ship into an aggregate message according to preset rules; using a data processor to obtain the aggregate message from a direct memory access unit through an interrupt control mechanism and store it in a data receiving buffer queue; retrieving the aggregate message from the data receiving buffer queue and parsing out each individual message and its corresponding timestamp, adding a PCAP file header to each individual message to form multiple PCAP format messages including the reception time; and storing all PCAP format messages according to configuration rules to form a PCAP format file. This invention enables effective storage of real-time message data from a marine data acquisition device when the real-time communication data traffic on the ship is too large, improving the efficiency and stability of the ship's message communication system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of electronic information technology for surface vessels, and in particular to a real-time message storage method and system for a shipboard data acquisition device. Background Technology

[0002] With the development of surface vessel construction technology, in order to ensure the safety and efficiency of surface vessel operation, ship status signal monitoring and real-time data acquisition devices have been widely used, leading to a sharp increase in the amount of real-time data in ship communication. This surge in real-time ship communication data places higher demands on the real-time message data storage of ship data acquisition devices. When the real-time communication data traffic is too high, if an effective real-time message storage strategy is not designed, it will result in incomplete data storage of the real-time messages in the data acquisition device, or even the device freezing or crashing. This hinders ship management and maintenance personnel from conducting fault analysis and anomaly monitoring of Ethernet equipment. Summary of the Invention

[0003] The main objective of this invention is to propose a real-time message storage method and system for a marine data acquisition device, which solves the problem of incomplete data storage of real-time messages in a marine data acquisition device, or even device crashes or freezes, when the real-time communication data traffic on ships is too large.

[0004] The technical solution adopted in this invention is: a real-time message storage method for a marine data acquisition device, comprising: Using programmable logic devices, multiple individual messages received by the ship are merged into an aggregate message according to preset rules; Using a data processor, aggregated messages are retrieved from the direct memory access unit via an interrupt control mechanism and stored in the data receive buffer queue. Extract aggregated messages from the data receive buffer queue, parse out each individual message and its corresponding timestamp, add a PCAP header to each individual message, and form multiple PCAP format messages including the receive time; All PCAP format messages are stored according to the preset configuration rules to form a PCAP format file.

[0005] According to the above technical solution, the programmable logic device includes an FPGA.

[0006] According to the above technical solution, the data processor includes a CPU network card, and the length of the aggregated message is less than or equal to the length of the CPU network card's maximum transmission unit.

[0007] According to the above technical solution, the specific methods for generating multiple PCAP format messages include: Extract the aggregated message from the data receive buffer queue and obtain the total number of individual messages aggregated in the aggregated message; Set the number of processed single messages to an initial value of 0, move the reading address starting from the first single message, read each single message one by one, parse the message length, timestamp when it was received and the original content of the currently read single message, and add a PCAP header to the currently read single message; Add the PCAP header to the data receive buffer queue and increment the total number of processed single messages by 1; When the total number of processed individual messages equals the total number of individual messages aggregated in the aggregated message, stop reading individual messages.

[0008] According to the above technical solution, the PCAP file header corresponding to a single message includes the message length of the single message and the timestamp when it was received.

[0009] According to the above technical solution, the configuration rules include storing all PCAP format messages, storing only the Layer 2 protocol headers of PCAP format messages, storing only the IP protocol headers of PCAP format messages, storing only the UDP / TCP protocol headers of PCAP format messages, and storing PCAP format messages of a custom length.

[0010] According to the above technical solution, the data receiving buffer queue has a ring storage structure.

[0011] Another aspect of the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the steps of the above-described real-time message storage method for a marine data acquisition device.

[0012] Another aspect of the present invention provides a computer storage medium storing a computer program executable by a processor, the computer program executing the above-described real-time message storage method for a marine data acquisition device.

[0013] Another aspect of the present invention provides a real-time message storage system for a marine data acquisition device, including a message aggregation module, a data receiving module, a message parsing module, and a message storage module; The message aggregation module is used to combine multiple single messages received by the ship into an aggregated message according to preset rules using a programmable logic device. The data receiving module is used to obtain the aggregated message from the direct memory access unit through the interrupt control mechanism using the data processor, and store it in the data receiving buffer queue. The message parsing module is used to extract aggregated messages from the data receiving buffer queue and parse out each individual message and its corresponding timestamp, add a PCAP file header to each individual message, and form multiple PCAP format messages including the receiving time. The message storage module is used to store PCAP format messages.

[0014] The beneficial effects of this invention are as follows: This invention provides a real-time message storage method and system for a marine data acquisition device. First, multiple individual messages that the ship is about to receive are merged into a single aggregated message, reducing the resources occupied by redundant data and improving the overall efficiency and stability of the ship's message communication system. Next, the received aggregated message is stored in a data receiving buffer queue, re-parsed into individual messages, and encapsulated into a PCAP format file that can be recognized by packet capture software. This solves the problem of incomplete real-time message data storage or even device crashes when the real-time communication data traffic on ships is too high, achieving effective storage of ship communication data under high-traffic, high-speed environments.

[0015] Furthermore, the present invention adopts flexible configuration rules when storing all PCAP format messages into PCAP format files, thereby improving the flexibility of real-time message storage.

[0016] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the real-time message storage method of the data acquisition device according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the merging of multiple individual messages into an aggregated message in the real-time message storage method of the data acquisition device according to an embodiment of the present invention; Figure 3 This is a logical diagram of the data acquisition device real-time message storage method of the present invention for forming multiple PCAP format messages; Figure 4 This is a schematic diagram illustrating the formation of a PCAP format file in the real-time message storage method of the data acquisition device according to an embodiment of the present invention; Figure 5 This is a structural diagram of the real-time message storage system of the data acquisition device according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] It should be noted that the illustrations provided in the embodiments of the present invention are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In this invention, it should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used only for descriptive and distinguishing purposes and should not be construed as indicating or implying relative importance.

[0022] Example 1 This embodiment provides a real-time message storage method for a marine data acquisition device, the process of which is as follows: Figure 1 As shown, the steps include: S1. Using programmable logic devices, multiple individual messages received by the ship are merged into an aggregate message according to preset rules. This process is as follows: Figure 2 As shown in the figure, the left side of the figure shows the format of a single message before merging, and the right side shows the format of the aggregated message after merging. The FPGA hardware packages multiple SPACs (single messages) into MPACs (aggregated messages) according to rules. By reducing duplicate format fields, such as magic words and basic identifiers that are present in each single message, the total amount of data is reduced, and the storage pressure is reduced.

[0023] This step is performed in an environment where shipboard real-time communication data transmission speed is high and the transmission traffic is large. Merging multiple individual messages into an aggregated message effectively reduces the complexity of subsequent storage tasks. In this embodiment, the shipboard communication network data traffic rate reaches 800M / s.

[0024] Furthermore, the programmable logic device includes hardware such as an FPGA.

[0025] S2. Using the data processor, the aggregated message is obtained from the direct memory access unit through the interrupt control mechanism and stored in the data receive buffer queue.

[0026] In this embodiment, the data processor is selected as a CPU (Central Processing Unit), and the maximum transmission unit length of its network card is 9600 bytes. Therefore, the length of the aggregated packet should be less than or equal to 9600 bytes.

[0027] Furthermore, the data receiving buffer queue has a ring storage structure.

[0028] S3. Extract aggregated messages from the data receive buffer queue, parse out each individual message and its corresponding timestamp, add a PCAP file header to each individual message, and form multiple PCAP format messages including the receiving time.

[0029] Furthermore, the execution logic for generating multiple PCAP format messages is as follows: Figure 3 As shown, the steps include: T1. Extract the aggregated message from the data receiving buffer queue and obtain the total number of individual messages aggregated in the aggregated message.

[0030] T2. Set the number of processed single messages to an initial value of 0. Starting from the first single message, move the reading address and read each single message one by one. Parse the message length, timestamp when it was received, and original content of the currently read single message, and add a PCAP header to the currently read single message.

[0031] In this embodiment, in order to obtain the original content of a single message from the aggregated message, the timestamp of the single message should also be obtained, which is used to add to the PCAP file header to record the precise time of message reception.

[0032] T3. Add the single message with the PCAP header to the data receive buffer queue and increment the total number of processed single messages by 1.

[0033] T4. When the total number of processed individual messages equals the total number of individual messages aggregated in the aggregated message, stop reading individual messages. When the total number of processed individual messages equals the total number of individual messages aggregated in the aggregated message, all individual messages in the aggregated message have been read and processed, and reading ends at this point.

[0034] S4. Store all PCAP format messages according to the preset configuration rules to form a PCAP format file. The process of forming the PCAP format file is as follows: Figure 4 As shown.

[0035] Furthermore, the resulting PCAP format file can be read by packet capture software such as Wireshark, and the PCAP format file can be stored on a SATA hard drive.

[0036] PCAP is an industry-standard format for network data capture that can be directly parsed by tools such as Wireshark. This avoids compatibility issues caused by custom formats, allowing ship management and maintenance personnel to view and analyze data directly using general-purpose software without having to develop additional parsing tools, thus significantly reducing the technical threshold and usage costs.

[0037] Specifically, the PACP header consists of fields such as magic word, major version number, minor version number, time zone correction, precise timestamp, and link type. The magic word in the PACP header is used to identify the PACP file and determine the byte reading order. In this embodiment, the PACP header added to a single packet stores the timestamp and packet length at the time the packet was captured, i.e., the timestamp and data packet length copied from the packet parsing stage in step T3.

[0038] Furthermore, considering the different storage requirements for various messages, the configuration rules include storing all PCAP format messages, storing only the Layer 2 protocol header of PCAP format messages, storing only the IP protocol header of PCAP format messages, storing only the UDP / TCP protocol header of PCAP format messages, and storing PCAP format messages of custom length, which improves the flexibility of storage.

[0039] This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. When the computer-readable storage medium of this embodiment is executed by a processor, it implements the real-time message storage method of the marine data acquisition device in the method embodiment.

[0040] This embodiment also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, for performing the steps of the above-described real-time message storage method for a marine data acquisition device.

[0041] Example 2 This embodiment provides a real-time message storage system for a marine data acquisition device. This system executes the real-time message storage method for a marine data acquisition device described in Embodiment 1, and its structure is as follows: Figure 5 As shown, it includes a message aggregation module, a data receiving module, a message parsing module, and a message storage module.

[0042] The message aggregation module is used to combine multiple individual messages received by the ship into an aggregated message according to preset rules using programmable logic devices.

[0043] The data receiving module is used to obtain aggregated messages from the direct memory access unit through the interrupt control mechanism using the data processor, and store them in the data receiving buffer queue.

[0044] Furthermore, the data receiving buffer queue is declared by the data receiving module and released by the message parsing module.

[0045] The message parsing module is used to extract aggregated messages from the data receiving buffer queue and parse each individual message and its corresponding timestamp, add a PCAP file header to each individual message, and form multiple PCAP format messages including the receiving time.

[0046] The message storage module is used to store PCAP format messages.

[0047] The methods for implementing the functions of each module have been described in Example 1 and will not be repeated here.

[0048] In summary, the present invention provides a real-time message storage method and system for a marine data acquisition device, which can effectively store real-time message data of the marine data acquisition device when the real-time communication data traffic of the ship is too large, thereby improving the efficiency and stability of the ship message communication system.

[0049] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0050] The order of the steps in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0051] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A real-time message storage method for a marine data acquisition device, characterized in that, include: Using programmable logic devices, multiple individual messages received by the ship are merged into an aggregate message according to preset rules; Using a data processor, aggregated messages are retrieved from the direct memory access unit via an interrupt control mechanism and stored in the data receive buffer queue. Extract aggregated messages from the data receive buffer queue, parse out each individual message and its corresponding timestamp, add a PCAP header to each individual message, and form multiple PCAP format messages including the receive time; All PCAP format messages are stored according to the preset configuration rules to form a PCAP format file.

2. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The programmable logic device includes an FPGA.

3. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The data processor includes a CPU network interface card (NIC), and the length of the aggregated message is less than or equal to the length of the CPU NIC's maximum transmission unit (MTU).

4. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The specific methods for generating multiple PCAP format messages include: Extract the aggregated message from the data receive buffer queue and obtain the total number of individual messages aggregated in the aggregated message; Set the number of processed single messages to an initial value of 0, move the reading address starting from the first single message, read each single message one by one, parse the message length, timestamp when it was received and the original content of the currently read single message, and add a PCAP header to the currently read single message; Add the currently read single message after adding the PCAP header to the data receive buffer queue, and increment the total number of processed single messages by 1; When the total number of processed individual messages equals the total number of individual messages aggregated in the aggregated message, stop reading individual messages.

5. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The PCAP header corresponding to each message includes the message length and the timestamp when it was received.

6. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The configuration rules include storing all PCAP format messages, storing only the Layer 2 protocol headers of PCAP format messages, storing only the IP protocol headers of PCAP format messages, storing only the UDP / TCP protocol headers of PCAP format messages, and storing PCAP format messages of a custom length.

7. The real-time message storage method for the marine data acquisition device according to claim 1, characterized in that, The data receiving buffer queue has a ring storage structure.

8. A computer device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the steps of the real-time message storage method for a marine data acquisition device according to any one of claims 1-7.

9. A computer storage medium, characterized in that, It contains a computer program that can be executed by a processor, which performs the real-time message storage method of the marine data acquisition device according to any one of claims 1-7.

10. A real-time message storage system for a marine data acquisition device, characterized in that, It includes a message aggregation module, a data receiving module, a message parsing module, and a message storage module; The message aggregation module is used to combine multiple single messages received by the ship into an aggregated message according to preset rules using a programmable logic device. The data receiving module is used to obtain the aggregated message from the direct memory access unit through the interrupt control mechanism using the data processor, and store it in the data receiving buffer queue. The message parsing module is used to extract aggregated messages from the data receiving buffer queue and parse out each individual message and its corresponding timestamp, add a PCAP file header to each individual message, and form multiple PCAP format messages including the receiving time. The message storage module is used to store PCAP format messages.