An artificial intelligence-based ship engine room monitoring and alarm system

Abstract

This invention discloses an artificial intelligence-based ship engine room monitoring and alarm system, belonging to the technical field of monitoring and alarm systems. It comprises an edge intelligent gateway deployed within the ship's engine room, multi-source sensor terminals installed throughout the engine room, and a shore-based cloud service platform. The edge intelligent gateway connects to the multi-source sensor terminals via wired or wireless means; it receives data from multi-source heterogeneous sensors, performs protocol parsing and anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation; and it performs multi-level triggered frame extraction and intelligent analysis on video data with load adaptation. This invention introduces a configurable feature cross-layer before multimodal data fusion to generate derived features with clear physical meaning, such as temperature × CO concentration and smoke density / flame area. This provides subsequent AI models with more correlated and discriminative input information, enabling them to more sensitively capture early, weak fire coupling signs that are difficult to identify with a single sensor.

Description

Technical Field

[0001] This invention relates to the field of monitoring and alarm system technology, specifically to an artificial intelligence-based ship engine room monitoring and alarm system. Background Technology

[0002] The engine room is the core of a ship's power system. Its internal environment is complex and harsh, with high temperatures, high pressures, flammable and explosive materials, and strong electromagnetic interference. It is a high-risk area for major safety accidents such as fires. Therefore, real-time and accurate monitoring and early warning of the engine room environment are the core links to ensure the safety of ship navigation.

[0003] Existing systems often rely on single-type sensors (such as heat or smoke detectors) for independent judgment, lacking an effective mechanism for fusing multi-source heterogeneous data (such as video, temperature, various gas concentrations, current, etc.). The data from various sensors are isolated in terms of protocols and timing, forming information silos. This makes it impossible to capture the early, weakly coupled characteristics of a fire (such as the simultaneous slow rise of temperature and CO concentration), leading to false alarms and the potential to miss real hazards when facing complex environmental interference.

[0004] Current solutions place the main computational and analytical tasks on cloud servers, with edge devices on the ship's engine room only responsible for data acquisition and uploading. However, during long-distance ocean voyages, satellite network connections are expensive, have limited bandwidth, and are extremely unstable. This architecture can cause real-time alarm functions to fail during network outages or excessively high latency, posing serious security risks. Furthermore, continuously uploading unprocessed raw video streams consumes a significant amount of valuable bandwidth and exacerbates the risk of data breaches.

[0005] Furthermore, even if some systems employ edge computing units, their data processing methods are relatively simple. For example, in terms of data desensitization, they often use methods such as mosaicking fixed areas, which cannot intelligently identify and protect sensitive information that dynamically appears in the cabin (such as crew faces, instrument readings, equipment nameplates, etc.). Often, while protecting privacy, they destroy key image features used for fire analysis, failing to meet the needs of both privacy security and intelligent analysis.

[0006] Based on this, the present invention designs an artificial intelligence-based ship engine room monitoring and alarm system to solve the above problems. Summary of the Invention

[0007] In view of the above-mentioned shortcomings of the existing technology, the present invention provides an artificial intelligence-based ship engine room monitoring and alarm system.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] An artificial intelligence-based ship engine room monitoring and alarm system includes:

[0010] Edge smart gateways deployed in the ship's engine room, multi-source sensor terminals installed throughout the engine room, and a cloud service platform located on shore;

[0011] The edge intelligent gateway is connected to the multi-source sensor terminal via wired or wireless means, and communicates with the cloud service platform via the network;

[0012] The edge smart gateway is configured to perform the following operations:

[0013] It receives data from multiple heterogeneous sensors, performs protocol parsing, and performs anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation.

[0014] The system performs multi-level triggered frame extraction and intelligent analysis on video data with load adaptation to generate compressed video feature data packets.

[0015] Multimodal data is time-series aligned based on a dynamic time window, and feature cross terms are introduced for fusion to form a fused data frame;

[0016] Based on the fused data frame, local AI inference is performed to generate multi-dimensional alarm decision events;

[0017] Based on the real-time network status, the data transmission strategy is intelligently selected or a hierarchical caching strategy is adopted to send alarm decision events to the local alarm terminal in the cabin with the highest priority.

[0018] In collaboration with the cloud service platform, it enables secure model updates and data synchronization.

[0019] Furthermore, the protocol parsing and anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation includes: calling a pre-set standard protocol library and script engine to parse heterogeneous data, extracting sensor IDs and raw data values; using a hybrid PTP and NTP clock synchronization and combining it with a local clock drift compensation model trained based on historical data to compensate for the received timestamp; querying the sensor delay parameter table, calculating the actual occurrence time of the data, and generating standardized data objects.

[0020] Furthermore, the load-adaptive multi-level triggered frame extraction and intelligent analysis includes: real-time monitoring of NPU utilization and dynamic adjustment of video processing strategies based on predefined utilization thresholds; employing a multi-level triggered frame extraction mechanism based on background subtraction and changing thresholds; using a lightweight object detection neural network model to analyze video frames and output structured detection results; and concatenating and compressing the detection results, thumbnails, timestamps, and device IDs to generate video feature data packets.

[0021] Furthermore, the size of the dynamic time window is determined by the formula: window size = base window + K × sensor calibration delay, where K is a scaling factor.

[0022] Furthermore, the feature cross term includes feature combinations configured according to the fire physics mechanism, and the feature combinations include one or more of the following: the product of temperature and CO concentration, the ratio of smoke density to flame area, the product of current and temperature, and the product of oil fume concentration and base plate temperature.

[0023] Furthermore, the hierarchical caching strategy includes: writing fire alarm data to high-durability memory; writing early warning data to a memory protection zone with a wear leveling algorithm; writing normal data to a circular buffer; and periodically compressing and archiving historical data.

[0024] Furthermore, the edge intelligent gateway is configured to immediately send the highest priority alarm control command to the audible and visual alarms and crew monitoring terminals in the cabin via the local area network after generating a local alarm decision event.

[0025] Furthermore, the cloud service platform is configured to: receive data uploaded from the edge side for model training and optimization; generate a digitally signed incremental update package for the model and send it to the edge smart gateway; after the edge smart gateway verifies the signature, it tests the new model in a resource-sensitive shadow mode, and completes hot switching after the performance is stable.

[0026] Furthermore, the edge smart gateway is also configured to perform edge compliance desensitization processing before data is uploaded, by identifying sensitive areas in the image through an integrated lightweight semantic segmentation model, and then irreversibly erasing or blurring them.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects:

[0028] 1. This invention introduces a configurable feature cross-layer before multimodal data fusion to automatically generate derived features with clear physical meaning, such as the product of temperature and CO concentration, the ratio of smoke density to flame area, the product of current and temperature, and the product of oil fume concentration and bottom plate temperature. This provides subsequent AI models with more correlated and discriminative input information, enabling them to more sensitively capture early fire weak coupling signs that are difficult to identify with a single sensor, thereby effectively reducing the false alarm rate and false alarm rate of the system.

[0029] 2. This invention pushes the core AI reasoning and decision-making capabilities down to the edge intelligent gateway, enabling the system to still perform high-precision fire analysis in real time based on local computing even when the network is completely interrupted or the performance is poor, and immediately activate the audible and visual alarm in the engine room. This completely overcomes the strong dependence of the traditional centralized architecture on the cloud and network, and provides continuous safety assurance for ships in weak network scenarios such as ocean voyages.

[0030] 3. The PTP and NTP hybrid synchronization adopted in this invention, combined with a local clock drift compensation model trained based on historical data and a sensor delay parameter table, can effectively resist the interference of the harsh electromagnetic environment in the cabin on the clock signal, calculate the true occurrence time of the data, lay a solid foundation for the accurate fusion and subsequent analysis of multi-source data, and break the time silos between different sensors.

[0031] 4. By monitoring NPU utilization in real time and setting multiple threshold levels, the system can dynamically and adaptively adjust video processing strategies (such as reducing resolution and pausing non-core computing). In high-load scenarios, it prioritizes the resource supply for the core fire identification algorithm, intelligently balances computing accuracy and processing real-time performance, avoids system crashes or key frame loss due to computing power overload, and ensures continuous and stable system operation.

[0032] 5. This invention intelligently extracts frames from massive video streams at the edge and compresses them into extremely small structured feature data packets (including detection results, thumbnails, etc.), which are then uploaded only when the network is good. This greatly reduces the transmission pressure on expensive bandwidth such as ship satellite links, while also reducing the processing and storage load on cloud servers.

[0033] 6. The multi-source fusion reasoning mechanism of this invention enables the system to continue to perform fusion analysis and judgment based on the remaining normal sensor data when faced with occasional failures or data loss of one or more sensors, maintaining a certain level of fire monitoring capability and preventing the system from completely failing until the fault is repaired, thus enhancing the overall reliability of the system. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0035] Figure 1 This is a schematic diagram illustrating the configuration and execution process of an edge intelligent gateway in a ship engine room monitoring and alarm system based on artificial intelligence, according to the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0037] Reference Figure 1 As shown, this embodiment discloses an artificial intelligence-based ship engine room monitoring and alarm system, including:

[0038] Edge smart gateways deployed in the ship's engine room, multi-source sensor terminals installed throughout the engine room, and a cloud service platform located on shore;

[0039] The edge smart gateway connects to the multi-source sensor terminal via wired or wireless means and communicates with the cloud service platform via a network.

[0040] The edge smart gateway is configured to perform the following operations:

[0041] S1: Receives data from multiple heterogeneous sensors, performs protocol parsing, and performs anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation;

[0042] S101: Physical connection and protocol pre-configuration. The edge intelligent gateway establishes physical connections with various sensors in the cabin through its integrated RS-485, CAN bus, Ethernet port and ZigBee / Wi-Fi wireless module. The gateway file system pre-stores parsing libraries for standard protocols such as Modbus RTU / TCP, ONVIF, and NMEA 2000, and pre-configures corresponding Lua parsing scripts for each known model of proprietary protocol sensor. These scripts explicitly define their frame structure, data field offset, byte order and CRC check algorithm.

[0043] S102: Multi-protocol adaptive parsing and data extraction. After the gateway is powered on and started, the drivers for each communication interface are loaded and the port data is continuously monitored.

[0044] S1021: Packet capture and frame header recognition. When a packet arrives at any port, the gateway reads the packet's preamble (frame header) and matches it with the built-in standard protocol signature library.

[0045] S1022: Standard protocol parsing. If the frame header matches successfully, the corresponding standard protocol parser is called to extract the sensor ID and raw data value from the payload of the data packet.

[0046] S1023: The dynamic script parsing engine starts. If the frame header cannot match any standard protocol, the gateway starts the embedded Lua script engine, executes the protocol parsing script written in advance for the device, and outputs the sensor ID and raw data value.

[0047] This example uses a proprietary CO concentration sensor commonly used in aircraft cabins. Its frame structure is fixed at 5 bytes: the first byte is the frame header (fixed to 0xAA), the second byte is the sensor ID (range 0x01-0x30), the third and fourth bytes are the CO concentration data (stored in big-endian order, unit: ppm), and the fifth byte is the CRC checksum (the checksum is obtained by XORing the first 4 bytes with a 0x8005 polynomial). The corresponding Lua parsing script logic is as follows:

[0048] First, check if the first byte of the received data packet is 0xAA; otherwise, it is considered an invalid frame.

[0049] If so, extract the second byte as the sensor ID, multiply the third byte by 256 and add the fourth byte to convert it into a decimal CO concentration value, then calculate the CRC value of the first 4 bytes of data and compare it with the fifth byte. If the verification passes, output the sensor ID and CO concentration value; if the verification fails, mark it as data abnormal.

[0050] S103: Anti-interference hybrid synchronization and dynamic compensation. Upon startup, the gateway synchronizes with a high-precision clock source (such as an industrial camera) within the cabin that supports the PTP protocol, calibrating its local clock accuracy to the microsecond level. For sensors that do not support PTP, software-level time synchronization is performed via the NTP protocol. To overcome the reflection and attenuation interference of the cabin's metal structure on the clock signal, the gateway incorporates a local clock drift compensation model. This model, based on historical synchronization data, calculates a linear regression compensation parameter (including slope) for each sensor channel. and intercept The gateway timestamps the received data packet for each successfully parsed packet. For example, calculating the timestamp after interference suppression:

[0051]

[0052] Where the slope The value range is 0.998-1.002, and the intercept is... The value range is -5μs to +5μs. This parameter range is determined based on the electromagnetic interference characteristics generated by generators, motors, and other equipment in the engine room. The clock drift caused by this type of interference is usually on the order of a tiny magnitude. A slope close to 1 can avoid overcompensation. A microsecond-level intercept can cover the time deviation under most interference scenarios. It can be finely adjusted according to the electromagnetic interference intensity of the sensor's installation location (e.g., for sensors near generators). Take 1.001-1.002, (Take a range of -3μs to -1μs).

[0053] The gateway has a built-in sensor delay parameter table that stores the initial values ​​of the inherent response delay of each sensor model and calculates the actual occurrence time of the physical quantity.

[0054]

[0055] in, This is the delay value for the sensor ID. Ultimately, all data is encapsulated into a unified, standardized data object, including the sensor ID, The data value is then sent to the raw data cache queue.

[0056] The delay values ​​stored in the sensor delay parameter table are typical inherent response delay values ​​obtained in advance by consulting the factory calibration documents and technical specifications of various sensors;

[0057] In other implementations, for sensors whose delay values ​​cannot be directly obtained from documentation, the delay value is determined through experimental calibration. When the gateway is running, it can query the corresponding sensor calibration delay value from the parameter table using the sensor ID.

[0058] It should be noted that the aforementioned hybrid synchronization specifically refers to the hybrid clock synchronization scheme adopted by the edge smart gateway, which combines hardware-level synchronization and software-level synchronization. Specifically, it has two meanings:

[0059] The protocol is hybrid, supporting both PTP (Precision Time Protocol) for microsecond-level synchronization with high-precision devices that support hardware timestamps (such as industrial cameras) and NTP (Network Time Protocol) for millisecond-level software synchronization with ordinary sensors.

[0060] The compensation hybrid approach combines a clock drift compensation model based on historical data and sensor inherent delay compensation with software synchronization.

[0061] S2: Performs multi-level triggered frame extraction and intelligent analysis on video data with load adaptation to generate compressed video feature data packets; and accurately fuses them with time-series sensor data.

[0062] S201: Data classification and routing. The data preprocessing module retrieves standardized data objects sequentially from the raw data cache queue and routes them to different processing sub-threads according to the sensor ID prefix. Video stream data is sent to the video processing channel, and parameter data such as temperature and gas are sent to the timing signal processing channel.

[0063] S202: Load-adaptive multi-level triggered frame dropping and intelligent analysis;

[0064] S2021: Resource monitoring and dynamic degradation. After the video processing channel is initialized, the NPU utilization is monitored in real time. The system sets a predefined utilization threshold to dynamically adjust the video processing strategy to balance computing load and processing real-time performance.

[0065] The first threshold for NPU utilization is set at 60%, and the second threshold is set at 85%. These thresholds are determined based on the computing power characteristics of commonly used edge gateway NPUs such as RK3588 and Jetson Nano. Below 60% is considered a range with sufficient computing power, which can support full-function operation; above 85% is a high-load threshold, and a degradation strategy needs to be initiated to avoid downtime.

[0066] When the NPU utilization rate is below the first threshold, it indicates that the computing resources are sufficient, the system is in the default mode, and executes all preset video processing steps.

[0067] When the NPU utilization reaches or exceeds the second threshold, it indicates that computing resources are strained, and the system immediately and automatically enters overload protection mode. In this mode, the system prioritizes the core fire detection function and adopts the following degradation strategy to reduce the computing load:

[0068] The processing resolution of all video streams is reduced from high resolution to a lower resolution in order to strike a balance between computational load and target detection accuracy;

[0069] Suspend non-core background difference calculations;

[0070] Reduce the frame rate to the minimum baseline level required to maintain basic monitoring functions.

[0071] To ensure that the system does not crash or lose critical video frames (such as frames that may contain fire alarm signals) under high load conditions, priority is given to ensuring the resource supply for the core fire identification algorithm, thereby ensuring the continuity and reliability of the alarm system under high load scenarios and indirectly improving the overall alarm accuracy.

[0072] S2022: Multi-level trigger frame extraction. In the default mode, the video processing channel adopts a multi-level trigger mechanism to optimize resource utilization. First, the video stream is subjected to low-complexity background difference calculation. If the difference between consecutive frames exceeds the preset threshold dynamically adjusted according to the scene, it indicates that there is a significant change in the picture, which may be a potential fire. Then, full-resolution frame capture and analysis are immediately triggered. Under normal conditions without significant changes, the system extracts video frames at a lower frequency that can meet the basic monitoring continuity requirements in order to save computing resources.

[0073] S2023: Decoding and feature extraction. The extracted image frames are decoded into RGB format and scaled to 512x288 pixels. The frames are then fed into a lightweight object detection neural network based on the NanoDet architecture and quantized with INT8. The network outputs a structured JSON array containing the detected object category, confidence score, and bounding box coordinates.

[0074] S2024: Structured data packet construction. The system will construct the structured detection results output in S2023 and the low-resolution thumbnail (compressed by JPEG) corresponding to the frame. The four elements, camera ID, etc., are concatenated into a long string in a predefined order and with a separator, and then compressed using the ZLIB algorithm to generate a very small video feature data packet.

[0075] S3: Perform time-series alignment of multimodal data based on dynamic time windows, and introduce feature cross terms for fusion to form a fused data frame;

[0076] S301: Timing data filtering and dynamic alignment. The timing signal processing channel performs moving average filtering on parameters such as temperature and gas concentration to eliminate instantaneous noise. This channel maintains a dynamically adjusted time window based on the system clock. The window size is positively correlated with the calibration delay of the associated sensor (window size = base window + K × sensor calibration delay), where:

[0077] The default window size is 2 seconds, used to accommodate basic time jitter generated during sensor data acquisition, transmission, and processing. K is a proportionality coefficient, ranging from 0.5 to 1.0, specifically set according to the differences in sensor response delay: for sensors with short response delays (usually ≤1 second), such as temperature sensors and current sensors, K is set to 0.5 to avoid data redundancy due to an excessively large window; for sensors with longer response delays (usually 1-3 seconds), such as smoke sensors and oil fume concentration sensors, K is set to 1.0 to ensure coverage of the complete data transmission cycle.

[0078] The base window is a fixed time constant, whose main function is to accommodate the base time jitter and uncertain delays generated during sensor data acquisition, transmission and processing;

[0079] K is a configurable scaling factor used to adjust the contribution of sensor delay to window size;

[0080] The sensor calibration delay is obtained by looking up the inherent response delay value of a specific sensor model from a preset sensor delay parameter table;

[0081] For each newly generated video feature data packet, in its Within the time window before and after a given moment, search for all other sensor data;

[0082] For intuitive understanding, in another implementation, for example, if the base window is 2 seconds, K is 0.5, and the gas sensor calibration delay is 10 seconds, the dynamic time window size is 2 + 0.5 × 10 = 7 seconds. The system will search for associated data points for the gas sensor within a 7-second range before and after the timestamp of the video feature data points.

[0083] S302: Feature Crossing and Data Frame Encapsulation. After extracting the original feature values ​​from the associated data, a feature crossing layer is added to generate physically meaningful cross features such as temperature × CO concentration and smoke density / flame area. The video feature data packet, all original features, and cross features are packaged into a unified multimodal fusion data frame. This data frame fully describes the comprehensive state of a certain area of ​​the cabin within a specific time period. For example:

[0084] Temperature × CO concentration: This cross term is designed based on the positive correlation between the fire heat release rate and the incomplete combustion product formation rate. In the early stages of a fire, an increase in temperature is often accompanied by an increase in the CO gas concentration due to incomplete combustion. The product of the two can amplify this coupling effect and is more sensitive to indicating early smoldering fires than using either feature alone.

[0085] Smoke density / flame area is designed to distinguish open flames from interfering sources (such as water vapor and dust). Open flames usually have high optical brightness and relatively low smoke density in their area, while a large amount of smoke is usually accompanied by a small flame area. This ratio helps to suppress false alarms of high smoke and low flame (such as dense smoke) or high flame and low smoke (such as lamp glare).

[0086] The cross-characteristic temperature × CO concentration and smoke density / flame area described above are merely illustrative examples. The characteristic cross layer of the present invention is a scalable architecture. In actual deployment, new sensors and corresponding cross items can be introduced according to the key protected areas and equipment in the cabin.

[0087] In another embodiment, current sensors are added to the electrical equipment areas such as distribution cabinets and motors. In order to monitor the risk of electrical fires, a current × temperature cross term can be added to the characteristic cross layer. The design basis of this cross term is that the conductor heats up due to overload or poor contact, and its heating power is proportional to the square of the current. The current × temperature term can be used as an effective approximation of this physical relationship and is used to sensitively capture the precursors of abnormal heating.

[0088] Adding oil fume concentration sensors below equipment such as gearboxes and hydraulic systems can monitor the risk of fire caused by lubricating oil leakage encountering high-temperature surfaces. An oil fume concentration × base plate temperature cross term can be added. The design basis of this cross term is that an abnormal increase in oil fume concentration indicates a possible leak. When it is combined with a high-temperature surface (such as a smoke exhaust pipe), the fire risk increases sharply. This product term can be used to quantify this coupled risk.

[0089] Those skilled in the art will understand that whether it is a flammable gas leak on an LPG carrier or a lithium battery thermal runaway on a container ship, new cross terms can be derived and configured into the system based on their disaster-causing mechanisms and the same design principles mentioned above (such as A×B characterizing the coupling effect and A / B used to distinguish interference). Therefore, this step is not listed in detail.

[0090] S4: Perform local AI inference based on the fused data frame to generate multi-dimensional alarm decision events;

[0091] S401: Feature vector construction. Extract all feature values ​​from the multimodal fusion data frame, including original features and cross features, and arrange them in a fixed order to construct a one-dimensional feature vector.

[0092] S402: Local multi-task AI model inference, inputting feature vectors into a pre-installed, lightweight, fully connected neural network classification model. The model outputs a multi-dimensional label, including: three-level classification probabilities (normal, warning, fire alarm), fire type probabilities (electrical, fuel, etc.), and a spread rate index.

[0093] S403: Decision and Event Generation. The system selects the category with the highest probability as the judgment result for the current state. If the probability value exceeds the activation threshold preset for that category, an alarm event is determined to have occurred, and a corresponding local alarm decision event is generated. This event includes the alarm level, alarm time, trigger source device identifier, confidence level, and associated fused data frame index.

[0094] The activation thresholds are specifically set as follows: 0.8 for fire alarm category, 0.6 for early warning category, and no threshold for normal category (normal is judged by default when the probability is highest). These thresholds are determined based on the need to balance false alarms and missed alarms in engine room fire identification. A fire alarm threshold higher than 0.5 can filter low-confidence interference signals, while a value of 0.8 can avoid false alarms caused by environmental noise and ensure that real fires are not missed. In practical applications, the threshold can be fine-tuned according to the aging of engine room equipment (e.g., the fire alarm threshold can be lowered to 0.75 on older ships to reduce the risk of missed alarms).

[0095] The activation threshold is set based on the following technical logic:

[0096] The probability value output by the AI ​​model reflects its confidence in the classification. In binary classification tasks, 0.5 is the theoretical classification decision boundary, indicating that the model cannot make a clear judgment.

[0097] By setting the activation threshold for alarm categories (such as fire alarms) to a value significantly higher than 0.5, a decision buffer is constructed to filter out low-confidence inferences caused by environmental interference, data noise, and other factors, ensuring that the system only triggers alarms when there is sufficient confidence.

[0098] Those skilled in the art should understand that the activation threshold can be flexibly configured within a range reasonably higher than the level of random guessing, depending on the different tolerance levels for false alarms and false misses in actual applications. For example, in some implementations, a value between 0.6 and 0.9 can be selected as the fire alarm determination threshold to achieve an appropriate balance between system sensitivity and reliability.

[0099] S5: Intelligently selects data transmission strategy or adopts hierarchical caching strategy based on real-time network status, and sends alarm decision events to the local alarm terminal in the cabin with the highest priority.

[0100] S501: Network status awareness, the gateway network module periodically sends heartbeat packets to the cloud to monitor network round-trip latency and packet loss rate in real time;

[0101] S502: Intelligent transmission strategy scheduling, specifically including the following steps:

[0102] S5021: When the network performance index is better than the first network quality threshold, the network connection is determined to be good, and the multimodal fusion data frame and local alarm decision event are uploaded to the cloud completely and in real time.

[0103] S5022: When the network performance index is between the first network quality threshold and the second network quality threshold, it is determined that the network connection is poor. The adaptive fault-tolerant differential transmission mechanism is activated, the difference between the current data frame and the previous successfully transmitted frame is calculated, the difference data is grouped according to its size, and forward error correction coding is performed on each group to generate redundant data packets, which are then uploaded to the cloud to enhance the data transmission's ability to resist packet loss.

[0104] The network quality thresholds are specifically set as a first network quality threshold (round-trip delay ≤ 100ms and packet loss rate ≤ 1%) and a second network quality threshold (round-trip delay ≤ 500ms and packet loss rate ≤ 5%). These thresholds are determined in conjunction with the typical characteristics of satellite networks commonly used on ocean-going vessels.

[0105] In terms of bandwidth, ship satellite networks typically range from 2 to 10 Mbps, which can support the transmission requirements of multimodal data (including compressed video feature packets) of this system and avoid bandwidth overload.

[0106] In terms of latency and packet loss rate, the first network quality threshold is an optimization index that can be achieved in near-shore, port or sea areas with strong satellite signals. It is suitable for scenarios with extremely high requirements for data real-time performance (such as uploading abnormal data of key equipment in the cabin to the cloud in real time to assist shore-based real-time monitoring).

[0107] The second network quality threshold is the level that satellite networks can normally achieve during ocean voyages. Although a 500ms delay cannot meet the needs of high-frequency real-time interaction, it can cover the core functions of this system (such as alarm event synchronization and non-urgent data caching and retransmission), and a packet loss rate of less than 5% can be compensated by the forward error correction coding mechanism.

[0108] S5023: When network performance indicators fall below the second network quality threshold, the network is judged to be interrupted or unavailable, and a high-reliability hierarchical caching mechanism is activated. All data is cached hierarchically based on its AI judgment results, specifically including:

[0109] Fire alarm data is written to high-durability memory (using industrial-grade eMMC5.1, whose erase / write life is ≥3000 times to ensure that critical data is not lost when power is off).

[0110] Write the early warning data into a protected memory area with a wear leveling algorithm (using NOR Flash with a capacity of ≥128MB, reducing local wear through block rotation to meet the medium protection requirements of early warning data).

[0111] Normal data is written to a circular buffer (allocate 2GB of memory space, adopt a first-in-first-out mechanism, and automatically overwrite the oldest data when the buffer is full, balancing storage efficiency and resource consumption).

[0112] During off-peak business periods (such as when cabin equipment is operating at low load, typically between 2 and 4 a.m.), the system automatically compresses and archives historical alarm data from high-durability storage to a large-capacity storage area (compression format is ZIP, compression rate is set to 50%), freeing up front-end storage space and further extending the overall storage life of the system.

[0113] S503: The highest priority command is issued. Regardless of the network status, once the S403 generates a local alarm decision event, the gateway immediately sends the highest priority alarm control command to the audible and visual alarms and crew monitoring terminals in the engine room through the local area network, achieving zero alarm delay.

[0114] Intelligent transmission, highly reliable caching, and instant alarms manage all data communications, ensuring that critical alarm information is not lost under any network conditions.

[0115] S6: Work with the cloud service platform to achieve secure model updates and data synchronization;

[0116] S601: Resume interrupted transmission and data synchronization. When the gateway senses that the network has recovered from an interrupted or poor state to a good state, it automatically checks its local cache and transmits all cached data (fire alarm data is transmitted in batches according to the protection level) to the cloud server for archiving.

[0117] S602: Model security update and verification, specifically including the following steps:

[0118] S6021: Secure delivery. The cloud server generates a digital signature for the new model file (such as .tflite format) after training and optimization, and sends it to the gateway along with the incremental model update package. The gateway receives the update package after verifying that the signature is correct.

[0119] S6022: Resource-Sensitive Shadow Test is designed for environments where edge gateway computing power and memory resources are limited. It continuously monitors key resource indicators such as NPU utilization and memory usage of the edge smart gateway. When the new model runs as a shadow model, it does not perform full inference on all input data. Instead, it only triggers the complete forward inference and result recording of the new model when the difference between the inference results of the new and old models on the same input data exceeds a preset difference threshold. This avoids resource exhaustion and service interruption that may be caused by running two models in parallel, and realizes model performance evaluation under controllable resource conditions. During the test period, the system compares the accuracy and false alarm rate of the new and old models based on the triggered difference samples to decide whether to enable the new model.

[0120] The difference threshold is set at 15%, which means that when the difference between the probability of fire type for the same data between the old and new models exceeds 15%, or the alarm level is inconsistent, the new model will be triggered to complete the reasoning.

[0121] This threshold is determined by combining the redundancy characteristic of edge gateways that typically reserve 30% computing power. A 15% difference can effectively filter out small fluctuations in the model output, while ensuring that significant performance differences are captured, thus balancing test accuracy and resource consumption.

[0122] The difference threshold is set based on the following technical logic:

[0123] This threshold is used to strike a balance between edge resource constraints and model evaluation reliability. It is essentially a resource filter that ensures that only inference results that significantly reflect differences in model performance are recorded and analyzed.

[0124] If the threshold is set too low, even slight output fluctuations will trigger recording and comparison, which can easily cause the resource-sensitive mode to degenerate into the full test mode, exacerbating resource consumption.

[0125] If the threshold is set too high, it can only capture extreme difference samples, making it difficult to effectively evaluate the overall performance of the model after the update (especially subtle degradations), thus weakening the significance of the test.

[0126] Those skilled in the art should understand that the difference threshold is a configurable system parameter that can be dynamically adjusted based on the real-time resource status of the edge gateway and the granularity requirements of the model evaluation, while taking into account both noise suppression and difference capture capabilities.

[0127] S6023: Edge compliance desensitization processing. Before uploading the video feature data package, a lightweight semantic segmentation model can be integrated on the edge side to identify sensitive areas in the image (such as faces, dashboard readings) and irreversibly erase them with pixelation, retaining only machine-readable features related to fire analysis (such as flame ROI, feature vectors).

[0128] The lightweight semantic segmentation model is as follows:

[0129] Using a decoder structure (such as DeepLabV3 Lite based on MobileNetV3), after training, it can classify image pixels and accurately segment predefined sensitive areas such as faces, uniform badges, dashboard readings, and equipment nameplates.

[0130] The model is trained using a combination of publicly available general scene segmentation datasets (such as COCO) and a self-built cabin environment image dataset, giving it both general target recognition capabilities and the ability to recognize cabin-specific elements.

[0131] For all sensitive areas identified by the model, the system sets their pixel values ​​to fixed color blocks (such as black) or performs irreversible blurring (such as high-intensity Gaussian blur). The processing intensity is based on making sensitive information unrecognizable and unrecoverable by the naked eye, thus achieving irreversible desensitization. Fire-related feature areas include, but are not limited to, image areas that directly characterize the fire, such as flames, smoke, and high-temperature hot spots. The feature integrity of these areas is crucial for AI analysis, so they are excluded from the desensitization scope. Only background sensitive information outside the above areas is erased.

[0132] The edge intelligent gateway, as the core processing unit of the system, includes multi-protocol interfaces, an NPU computing unit (RK3588 or Jetson Nano model, meeting the needs of lightweight AI inference), and various storage options (eMMC, NOR Flash, DDR4 memory). The gateway is configured with the following functional modules:

[0133] The multi-protocol data access and anti-interference synchronization module is used to receive and parse various types of sensor data, and generate standardized data objects through a hybrid synchronization and dynamic compensation mechanism.

[0134] The load-adaptive video analytics module is used to monitor computing load and dynamically adjust video processing strategies. It generates video feature data packets through frame extraction and lightweight model analysis.

[0135] The multimodal fusion processing module is used to perform temporal alignment on multi-source data and introduce feature interaction terms to form a fused data frame;

[0136] The edge intelligent reasoning and decision-making module is used to perform local AI reasoning based on fused data and generate alarm decision events.

[0137] The intelligent transmission and hierarchical storage module is used to adaptively select data transmission strategies based on network conditions and perform differentiated cache management according to data level.

[0138] The cloud-edge collaboration and security update module is used to manage data synchronization with the cloud, model security updates, and resource-sensitive testing.

[0139] The multi-source sensor terminal includes various types of sensors such as image (2-megapixel industrial camera, supporting wide dynamic range, adapting to strong light / shade environment in the cabin), temperature (thermocouple sensor), gas (CO sensor, combustible gas sensor), and smoke (photoelectric smoke sensor), which are connected to the edge smart gateway via wired or wireless means.

[0140] The cloud service platform includes model training and optimization (using the TensorFlow framework, iteratively training based on historical data uploaded from the edge), global situation analysis (supporting the aggregation of engine room data from multiple ships to generate regional safety reports), and a data archiving unit (using distributed storage), which is used to receive edge data, perform model optimization and macro-analysis, and distribute updates.

[0141] The system works in concert with the above modules to achieve real-time intelligent monitoring and highly reliable alarms for the ship's engine room environment.

[0142] It should be noted that the training and preparation of the AI ​​models (including object detection models and semantic segmentation models) involved in this invention are part of the model construction process itself, and are not the inventive point of this invention. Those skilled in the art should understand that any equivalent object detection or semantic segmentation model structure that can meet the requirements of lightweight and low latency to achieve the above functions falls within the protection scope of this invention.

[0143] Lightweight object detection models (such as NanoDet) can be obtained through the following conventional methods:

[0144] Image data containing targets such as flames and smoke in the ship's engine room are collected or acquired to form a training set. This training set is used to train an open-source or commercial NanoDet model until the model converges (training iterations ≥ 10,000 times, loss function value ≤ 0.05). Then, a standard post-training quantization (PTQ) tool (such as TensorRT) is used to convert the trained FP32 model to INT8 precision to adapt it to edge computing units.

[0145] Those skilled in the art will understand that, based on existing publicly available model architectures and training methods, the preparation and optimization of the above-mentioned model can be completed without creative effort, and it can be used to implement the system described in this invention.

[0146] It should be further clarified that this invention claims protection for the application method and system integration of the lightweight semantic segmentation model in the specific scenario of desensitization of ship engine room edges, rather than the specific structure or training details of the model itself. Those skilled in the art can obtain a model capable of recognizing the sensitive region based on publicly available model architectures (such as DeepLab, UNet, etc.) and datasets (such as COCO, Cityscapes, etc.) through conventional training and optimization methods, and deploy it in the system described in this invention. Therefore, the specific implementation details of the model are not essential technical features of this invention and do not affect those skilled in the art from implementing this invention.

[0147] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An artificial intelligence-based ship engine room monitoring and alarm system, characterized in that, include: Edge smart gateways deployed in the ship's engine room, multi-source sensor terminals installed throughout the engine room, and a cloud service platform located on shore; The edge intelligent gateway is connected to the multi-source sensor terminal via wired or wireless means, and communicates with the cloud service platform via the network; The edge smart gateway is configured to perform the following operations: It receives data from multiple heterogeneous sensors, performs protocol parsing, and performs anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation. The system performs multi-level triggered frame extraction and intelligent analysis on video data with load adaptation to generate compressed video feature data packets. Multimodal data is time-series aligned based on a dynamic time window, and feature cross terms are introduced for fusion to form a fused data frame; Based on the fused data frame, local AI inference is performed to generate multi-dimensional alarm decision events; Based on the real-time network status, the data transmission strategy is intelligently selected or a hierarchical caching strategy is adopted to send alarm decision events to the local alarm terminal in the cabin with the highest priority. In collaboration with the cloud service platform, secure model updates and data synchronization are achieved; The load-adaptive multi-level triggered frame dropping and intelligent analysis includes: Real-time monitoring of NPU utilization in the edge smart gateway's NPU computing unit, and dynamic adjustment of video processing strategies based on predefined utilization thresholds; A multi-level triggering frame-skipping mechanism based on background difference and changing thresholds is adopted; A lightweight object detection neural network model is used to analyze video frames and output structured detection results. The detection results, thumbnails, timestamps, and device IDs are concatenated and compressed to generate a video feature data package.

2. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The protocol parsing and anti-interference time synchronization processing based on hybrid synchronization and dynamic compensation include: It calls the pre-built standard protocol library and script engine to parse heterogeneous data and extract sensor IDs and raw data values; A hybrid PTP and NTP clock synchronization method is used, combined with a local clock drift compensation model trained based on historical data, to compensate for the received timestamp. Query the sensor delay parameter table, calculate the actual occurrence time of the data, and generate a standardized data object.

3. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The size of the dynamic time window is determined by the following formula: Window size = Base window + K × Sensor calibration delay, where K is a scaling factor.

4. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The feature cross term includes feature combinations configured according to the physical mechanism of fire, and the feature combinations include one or more of the following: the product of temperature and CO concentration, the ratio of smoke density to flame area, the product of current and temperature, and the product of oil fume concentration and base plate temperature.

5. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The tiered caching strategy includes: Write fire alarm data to high-durability memory; Write the early warning data into a protected memory area with a wear leveling algorithm; Write normal data into the circular buffer; Historical data should be compressed and archived regularly.

6. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The edge intelligent gateway is configured to immediately send the highest priority alarm control command to the audible and visual alarms and crew monitoring terminals in the cabin via the local area network after generating a local alarm decision event.

7. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The cloud service platform is configured as follows: Receive data uploaded from the edge side for model training and optimization; Generate a digitally signed incremental update package for the model and send it to the edge smart gateway; After verifying the signature, the edge smart gateway performs a new model test in a resource-sensitive shadow mode, and completes the hot switch after the performance stabilizes.

8. The ship engine room monitoring and alarm system based on artificial intelligence according to claim 1, characterized in that, The edge smart gateway is configured to perform edge compliance desensitization processing before data is uploaded. It identifies sensitive areas in the image through an integrated lightweight semantic segmentation model and performs irreversible pixelation or high-intensity blurring on them.

Images