Marine unmanned aerial vehicle cabin multistage fire alarm system and method based on edge calculation

Abstract

The invention provides a ship unmanned aerial vehicle cabin multistage fire alarm system and method based on edge computing, and the system comprises an intelligent shipborne system and a shore-based cloud system, the intelligent shipborne system is disposed in a ship cabin, and has the functions of data collection, data abnormity perception, fire monitoring and multistage early warning; the method is composed of three levels of data acquisition, edge processing and fire decision making. The shore-based cloud system is a cloud end of a shore-based system arranged on the land, has the functions of transmitting and receiving distress signals, storing remote data and tracing large fire accidents, and is composed of a communication unit, a storage unit and an alarm unit. The intelligent ship-borne system is connected with the shore-based cloud system through the satellite communication technology, infrared thermal imaging and multi-source sensing fusion is achieved, the device has the edge processing capacity, the multi-level fire judgment logic and the shore-based remote traceability function, and the device is suitable for the closed, complex and unattended ship engine room environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of ship safety monitoring, and specifically relates to a multi-level fire alarm system and method for an unmanned ship cabin based on edge computing. Background Art

[0002] Currently, with the gradual improvement of relevant regulations and technical standards in the field of smart ships, smart ships have become the focus of current research. The smart engine room is an important component of smart ships and a vital support for the realization of smart ship operation.

[0003] With the rapid development of intelligent ship technology, ship power systems, propulsion systems, fuel lines, and high-voltage electrical equipment are becoming increasingly integrated and automated. This has reduced the frequency of human inspections and increased the complexity of engine room structures. While intelligent engine rooms reduce manpower requirements, they also introduce new safety hazards. If immediate action is not taken, a fire can quickly spread throughout the ship, causing irreversible property damage, environmental pollution, and even life-threatening consequences.

[0004] Existing ship engine room fire monitoring systems mostly use standalone smoke, gas, or temperature sensors. While these sensors can monitor temperature and gas content in the early stages of a fire, even when smoke and fire are absent, they lack image recording capabilities, making it difficult to visually reconstruct the fire scene. They are also susceptible to environmental interference and have strict requirements for their installation location. For example, in a high-humidity, high-salt engine room environment, smoke sensors can misinterpret moisture, dust, oil mist, and salt fog as smoke. Furthermore, smoke can accumulate on the top floor of the engine room due to the rising heat of the hot air. If the smoke sensor is installed too low or near a vent, it may not detect any smoke at all. Temperature sensors trigger an alarm when the temperature at the detection point rises above a set threshold. However, the equipment in ship engine rooms is inherently heat-generating, and temperature sensors may misinterpret high temperatures generated during normal operation as a fire. Furthermore, temperature sensors should not be installed near air inlets or cooling lines, otherwise the measured temperature data will significantly deviate from the actual temperature in the fire area. Furthermore, smoke, gas, and temperature sensors are point-based sensors that can only detect within a limited area near their installation location and cannot cover the entire cabin. Furthermore, these sensors collect data and output an electrical signal to generate a fire alarm, making it impossible to determine the fire's development trend within the designated area. Therefore, fire monitoring systems based solely on independent sensors have limitations in terms of comprehensive coverage, information integration, and alarm accuracy, making them unsuitable for fire warning in unmanned intelligent cabins.

[0005] With the development of computer science and technology, video monitoring systems can achieve continuous, multi-angle monitoring without blind spots. Currently, a few ships have introduced fire monitoring systems based on visible light or infrared cameras. However, these fire monitoring systems use a centralized processing structure, and a failure will cause the entire fire monitoring system to fail. Moreover, a single video monitoring system cannot detect the early stages of a fire, such as rising temperatures and increasing CO concentrations. Therefore, relying solely on visible light or infrared cameras is difficult to meet the fire monitoring needs of unmanned cabins. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a multi-level fire alarm system and method for ship unmanned cabins based on edge computing in response to the above-mentioned problems. The system organically combines infrared video fire monitoring technology with sensor fire monitoring technology (such as smoke sensors, gas sensors, etc.) to construct a fire monitoring system with multi-source information fusion. While maintaining the advantages of global perception and visual traceability of video monitoring technology, it fully utilizes the sensitivity of traditional sensors to detect tiny anomalies to achieve early monitoring of intelligent cabin fires.

[0007] The embodiment of the present application is implemented as follows: An embodiment of the present application provides a multi-level fire alarm system for an unmanned ship cabin based on edge computing, which is characterized in that it includes an intelligent shipborne system and a shore-based cloud system connected via satellite communication. The intelligent shipborne system is deployed inside the ship's cabin and consists of a data acquisition layer, an edge processing layer and a fire decision layer; the shore-based cloud system is a cloud of a shore-based system set on land, and consists of a communication unit, a storage unit and an alarm unit. The storage unit stores fire data transmitted by the fire decision layer, the communication unit performs data communication with the fire decision layer, and the alarm unit receives a fire distress signal sent by the fire decision layer and sends it to the shore-based system to alarm.

[0008] In some optional embodiments, the data acquisition layer includes an infrared thermal imaging sensor, a CO gas sensor and a smoke sensor; the edge processing layer includes a video data processing board and a sensor data processing board, the infrared thermal imaging sensor transmits data with the video data processing board, and the CO gas sensor and the smoke sensor both transmit data with the sensor data processing board; the fire decision layer includes a communication unit, a decision unit, a control unit and a storage unit that are interconnected, transmits data with the edge processing layer and the shore-based cloud system, determines the fire hazard level, and links the cabin alarm and fire protection system, and finally saves the data locally.

[0009] In some optional implementation schemes, the video data processing board and the sensor data processing board both include a central processing and system memory unit, a wifi wireless communication unit, a wired communication interface, a universal serial port interface, and a power and reset module. The central processing and system memory unit includes a main control chip and an internal memory to complete system startup boot, file system loading and edge AI model storage; the wifi wireless communication unit is an external Wi-Fi module, which establishes a wireless connection and communication with the data acquisition layer; the wired communication interface is a Gigabit Ethernet PHY chip, which is connected to the main control chip; the universal serial port interface is a serial port connected to the UART interface of the main control chip to complete debugging and module expansion connection; the power and reset module is an onboard DC-DC power supply, and an introduced watchdog protection circuit.

[0010] A multi-level fire alarm method for an unmanned cabin in a ship based on edge computing is characterized by comprising the following steps: Step a: Divide the ship's engine room into smaller sub-areas based on internal structural layout, equipment density, heat source distribution, and ventilation paths, ensuring that each sub-area has spatial independence and fire risk homogeneity. In each sub-area, multiple infrared thermal imaging sensors, CO gas sensors, and smoke sensors are placed and connected to the edge processing layer. In step b, the infrared thermal imaging sensor in the cabin sub-area is connected to the video data processing board via Wi-Fi, and transmits the infrared video data, including the thermal imaging video data and the temperature information of the coverage area, to the video data processing board via Wi-Fi. The CO gas sensor and the smoke sensor in the cabin sub-area are also connected to the sensor data processing board via Wi-Fi, and transmit the CO concentration value and the smoke concentration value at the detection point to the sensor data processing board via Wi-Fi. Step c: The edge processing layer analyzes and processes the infrared video data and sensor data acquired by the data acquisition layer, and sends abnormal fire data and fire risk scores based on the infrared video data and sensor data to the communication unit of the fire decision layer; In step d, the decision-making unit uses the fire data and fire risk scores sent by the edge processing layer to initiate corresponding linkage measures based on the fire level. It also conducts BP neural network training to optimize the weight parameters of thermal imaging data, temperature data, CO gas concentration, and smoke concentration in determining the fire hazard level, thereby improving the accuracy of fire prior judgment and achieving rapid response. In step e, if the communication unit of the shore-based cloud system receives the distress signal and fire abnormality data, the fire abnormality data is stored in the storage unit for backup to facilitate later accident tracing, and the alarm unit notifies the base station to arrange fire rescue for the ship.

[0011] In some optional embodiments, step c includes the following specific contents: The video data processing board analyzes and processes the infrared video data, that is, the video data processing board processes the temperature data and thermal imaging video data within the monitoring range of the infrared thermal imaging sensor; the sensor data processing board analyzes and processes the sensor data, that is, the sensor data processing board processes the gas concentration and smoke concentration data within the sensor detection range; when one of the video data processing board and the sensor data processing board first detects that the data it is responsible for processing is abnormal, it marks the area where the abnormal data is located and sends it to the other party via WiFi, while increasing its own sampling frequency and performing a fire judgment score on the abnormal area.

[0012] In some optional embodiments, the BP neural network training in step d uses a method combining a priori scorer of the BP neural network with a main model to reversely optimize the weight parameters of the priori scorer, specifically including the following: The prior scorer calculates the total fire risk score based on the received thermal imaging data, temperature data, CO gas concentration, and smoke concentration fire risk score. , the prior scorer is calculated as:

[0013] in, ; 、 、 、 is the prior weight parameter to be trained, b is the bias term, σ is the Sigmoid activation function, Reflects the credibility of the current edge anomaly; A BP neural network model containing four fire judgment features including thermal imaging image features, temperature vector, gas concentration vector and smoke concentration vector is constructed as the main fire judgment model to output the final fire hazard level. The present invention can truly reflect the reliability of the current edge node judgment. The joint loss function is constructed as follows:

[0014] in, is the classification loss of the main neural network, which is used to supervise the prediction accuracy of the final fire grade; is the loss term of the prior scorer, used for reverse training 、 、 、 ; The expected prior value set according to the true label is recorded as ,but ; The gradient descent optimization method with momentum is introduced in the training phase as follows:

[0015]

[0016] in, Represents trainable parameters 、 、 、 ; is the learning rate; is the momentum factor; is the joint loss function about gradient; is the weight update amount of the previous iteration.

[0017] In some optional embodiments, the video data processing board analyzes and processes the infrared video data as follows: The infrared thermal imaging video stream is divided into sequence frame images according to the timestamp, and each image frame contains temperature matrix information. Camera equipment at the time , , the image frame obtained is ; Infrared thermal imaging pictures Convert to grayscale image, grayscale processing method uses weighted average method:

[0018] in, 、 、 Represents the red, green and blue components in position The brightness value, 、 、 Represent the weights of the red, green and blue components in grayscale calculation respectively; In order to unify the dynamic range of the grayscale image and facilitate threshold discrimination, the grayscale image Normalization is performed to standardize the image grayscale dynamic range to [0, 255]. The calculation formula is as follows:

[0019] in, 、 Respectively represent the minimum and maximum grayscale values ​​in the current image; Set grayscale threshold , binarize the grayscale image, extract the high temperature candidate area, and obtain the flame mask image , which is defined as follows:

[0020] Using eight-neighborhood calculation method to image All the pixels with a value of 1 in the image are analyzed for connected domain extraction. connected regions, respectively , ,…, : ; Set the minimum fire source area judgment threshold to , using the formula Calculate the area of ​​each connected region ,like Greater than or equal to , indicating that the current frame has identified a suspected fire area, and the suspected fire area is marked as a key focus area , and set the fire confirmation flag value of the frame Set to 1 to start the subsequent multi-frame tracking detection mechanism; Continuously read the following frames from the current frame Frame images, and perform the above processing and analysis on each frame image, assuming that the continuous frame set is: , for each frame image in the frame set, repeatedly extract the flame mask image in the image , and record whether it is in the key focus area And the flame area meets the threshold condition. If there is a suspected fire area that meets the condition in the frame, the fire confirmation flag value of the frame is set to:

[0021] Where n=1,2,…,N; Calculate the fire risk score for the current detection period , which is calculated as follows:

[0022] in: , indicating infrared camera equipment exist The confidence score of fire events at all times; the higher the score, the more stable and continuous high temperature anomalies are in the area, and the higher the probability of fire.

[0023] In some optional implementation schemes, the video data processing board analyzes and processes the temperature data as follows: When there is no data anomaly, the video data processing board will first calculate the initial background temperature mean of each pixel by obtaining the temperature matrix of N consecutive frames. , and use this average as the normal temperature baseline for comparison with subsequent temperature rise values; secondly, obtain the temperature matrix in the new frame , using the formula Calculate the temperature difference between the new frame and the background, and determine whether there is a temperature rise. The judgment is based on: , is the temperature rise threshold; if the temperature rise condition is met, the eight-neighborhood calculation method is used to find the continuous temperature rise area , set the area threshold of the temperature rise area to If the area , it is considered that there is a suspected temperature rise area; then, the average temperature increment of the temperature rise area in the subsequent L frames is calculated: , and then use linear regression to calculate the slope , if the slope , indicating that the area is rapidly heating up, constituting an early fire warning; finally, combining the temperature rise amplitude, temperature rise rate, and temperature rise area parameters, a fire risk score based on temperature data is obtained. , the scoring criteria are as follows:

[0024] in, , , , The fire risk score is respectively the temperature rise amplitude, temperature rise rate and temperature rise area. Weight parameter in the calculation.

[0025] In some optional embodiments, the sensor data processing board performs edge processing and analysis on the CO concentration data and the smoke concentration data as follows: The CO gas sensor and smoke sensor array arranged in the cabin sub-area obtain the CO concentration and smoke concentration at each detection point. The sampling period is a fixed interval. , the concentration data collected in each sampling period is recorded as:

[0026] in, represents the concentration at the current moment, and n is the length of the historical time window of the concentration data; Set three concentration threshold points , and divide these three points into two continuous subintervals and , and are mapped to score intervals respectively and ; Set concentration score , concentration scoring function for:

[0027] in, , ; Represents the upper limit of safe concentration, Represents the starting point of the warning concentration, represents the upper limit of the map concentration; Represents the highest value of the current concentration score, Representative concentration reached Concentration score at the value; set up The time window step selected for analyzing the concentration growth trend is: For the current moment The concentration value of each sampling point, Indicates the concentration value n sampling periods ago, and the average growth rate of the concentration is calculated using the linear slope method :

[0028] And map the average growth rate of concentration to the concentration growth rate score , , mapping function is defined as:

[0029] in, The turning point of medium-speed growth is the warning threshold for rapid growth; represent The rating value at time The current concentration score Concentration growth trend score Linear weighting is performed to obtain a fire hazard score based on concentration :

[0030] in, , is the weight coefficient, satisfying , .

[0031] In some optional embodiments, when the BP neural network calculates the fire hazard level, The scores are mapped to fire hazard levels:

[0032] =1 means the danger level is 1, which means there is no open flame at present, but there is a fire risk. At this time, the control unit links the broadcast and alarm lights to issue a fire warning; =2 means the fire danger level is 2, indicating that an open flame has appeared. At this time, the control unit will link the broadcast and alarm lights to issue a fire alarm, and at the same time link the fire protection system to carry out targeted fire extinguishing. =3 represents the fire danger level of 3, indicating that the current fire is large and has a tendency to spread. At this time, the control unit links the broadcast and alarm lights to issue a fire alarm, links the fire protection system to carry out targeted fire extinguishing, and at the same time, the communication unit sends a fire distress signal and fire abnormality data to the shore-based cloud layer communication unit.

[0033] The beneficial effects of the present application are as follows: 1. The present application provides a multi-level fire alarm system for unmanned aircraft cabins in ships based on edge computing, which realizes edge intelligent perception and distributed fire judgment, improves the real-time performance of fire monitoring, and improves the overall response speed and monitoring accuracy of the fire monitoring system; 2. The video stream processing unit and the sensor data processing unit are functionally decoupled at the physical layer. When a hardware failure occurs in any data acquisition unit, the undamaged module still maintains the basic data acquisition and fire monitoring functions, which can reduce the risk of system paralysis caused by single point failure; 3. An intelligent fire hazard level decision-making mechanism based on a priori scorer and BP neural network is introduced in the fire decision-making layer. The data collected in the ship's engine room will be used as a training set, and during the training process, the weight parameters of various sensor data in the judgment of fire hazard levels will be automatically optimized to make it more consistent with the monitored ship's engine room environment and adaptable to various ship types; 4. Infrared thermal imaging image data and sensor data serve as trigger conditions for each other, improving the accuracy of monitoring the initial temperature rise stage of fire and small flames; 5. All fire event data will be locally stored in the fire decision-making layer, with visual backtracking capabilities to facilitate accident analysis; 6. The fire decision-making layer divides the fire hazard level into three levels, and each level has corresponding measures to improve the sensitivity, accuracy and system flexibility of ship unmanned cabin fire monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 This is an overall architecture diagram of the multi-level fire alarm system in an embodiment of the present application; Figure 2This is a structural diagram of the video data processing board and the sensor data processing board in the embodiment of the present application; Figure 3 A flowchart of a multi-level fire alarm in an embodiment of the present application; Figure 4 This is a flowchart of the fire level determination performed by the decision-making unit in an embodiment of the present application. DETAILED DESCRIPTION

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0037] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0038] It should be understood that the size of the serial numbers of the steps in the embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0039] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0040] The features and performance of the present application are further described in detail below with reference to the embodiments.

[0041] like Figure 1 As shown, the edge computing-based multi-level fire alarm system for ship unmanned cabins provided by the present application includes an intelligent shipborne system and a shore-based cloud system connected via satellite communication. The intelligent shipborne system is deployed inside the ship's cabin and consists of a data acquisition layer, an edge processing layer and a fire decision layer; the shore-based cloud system is a cloud of a shore-based system set on land, and consists of a communication unit, a storage unit and an alarm unit. The storage unit stores fire data transmitted by the fire decision layer, the communication unit communicates data with the fire decision layer, and the alarm unit receives the fire distress signal sent by the fire decision layer and sends it to the shore-based system to alarm.

[0042] The data acquisition layer collects infrared video data and sensor data from the ship's engine room. Infrared video data refers to infrared thermal imaging video data and temperature distribution data within the monitored area; sensor data refers to CO gas concentration and smoke concentration values ​​within the detected range. Therefore, this invention involves four sets of data: infrared thermal imaging video data, temperature distribution data, CO gas concentration values, and smoke concentration values.

[0043] The data acquisition layer includes infrared thermal imaging sensors, CO gas sensors, and smoke sensors. The infrared camera is an infrared thermal imaging sensor capable of simultaneously outputting thermal image sequences and temperature data. Its temperature data reflects the temperature of the monitored area, with each pixel in the thermal image corresponding to the surface temperature of an object at a specific spatial location within the monitored area. The gas sensor detects real-time changes in CO gas concentration within the cabin environment; the smoke sensor detects real-time changes in smoke particle concentration within the cabin environment. The data acquisition layer includes a Wi-Fi module, which enables data transmission with the edge processing layer via Wi-Fi wireless communication technology. Wi-Fi transmission is used because the cabin is home to numerous sensors, and wireless connectivity reduces wiring costs while also allowing for greater flexibility in installation locations.

[0044] The edge processing layer receives infrared video and sensor data from the data acquisition layer, preprocesses it, and makes preliminary fire assessments. If any abnormal data is detected, all data for that period, along with the four fire risk scores, is sent to the communication unit in the fire decision layer. The connection between the edge processing layer and the fire decision layer's communication unit is wired to ensure the real-time and integrity of the transmitted data.

[0045] The edge processing layer includes a video data processing board and a sensor data processing board. The infrared thermal imaging sensor transmits data with the video data processing board, and the CO gas sensor and smoke sensor transmit data with the sensor data processing board.

[0046] The fire decision layer consists of a communication unit, a decision unit, a control unit, and a storage unit, all interconnected and capable of communicating with each other. The communication unit is used by the fire decision layer to transmit data to the edge processing layer and the shore-based cloud layer. The decision unit uses the BP neural network to determine the fire hazard level based on the four fire data sources and fire risk scores received by the communication unit. The control unit is used to link the engine room alarm system and the fire protection system based on the fire hazard level. The storage unit is used to locally store abnormal data to facilitate investigation of the cause of the accident.

[0047] Example 1 like Figure 2As shown, in this embodiment, the video data processing board and the sensor data processing board both include a central processing and system memory unit, a wifi wireless communication unit, a wired communication interface, a universal serial port interface, and a power supply and reset module.

[0048] The central processing and system memory unit consists of a main control chip and internal memory. The main control chip uses the M300 chip as the main control chip, and is equipped with 256MB of LPDDR memory for running the system core and caching edge data analysis models. The M300 chip also integrates a QSPI Flash chip, which is connected through the M300's internal SPI Flash controller to complete tasks such as system boot, file system loading, and edge AI model storage.

[0049] The wifi wireless communication unit is an external Wi-Fi module that is connected to the M300 chip through the SDIO interface and establishes a wireless connection and communication with the data acquisition layer.

[0050] The wired communication interface is a Gigabit Ethernet PHY chip. The main control chip M300 is connected to the onboard Ethernet physical layer chip PHY (YT8521) through its built-in RGMII0 Ethernet interface. The PHY chip further connects to the Ethernet transformer through differential signals and outputs to the RJ45 standard Ethernet connector.

[0051] The universal serial port connects to the main control chip's UART interface for debugging and module expansion. The power and reset module provides power to the entire board through an onboard DC-DC power supply and incorporates a watchdog protection circuit to prevent system freezes. In harsh environments, the M300 acquisition and control board automatically resets if an abnormal fault occurs, ensuring system recovery.

[0052] Example 2 like Figure 3 As shown, the above-mentioned edge computing-based multi-level fire alarm system for unmanned aircraft cabins includes the following steps when used: First, the ship's engine room is divided according to the internal structure layout, equipment density, heat source distribution, and ventilation path to ensure that each sub-area has spatial independence and fire risk homogeneity. In each sub-area, multiple infrared thermal imaging sensors, CO gas sensors, and smoke sensors are placed and connected to the edge processing layer.

[0053] 2. The infrared thermal imaging sensor in the cabin sub-area is connected to the video data processing board via Wi-Fi, and transmits the infrared video data, including thermal imaging video data and temperature information of the coverage area, to the video data processing board via Wi-Fi. The CO gas sensor and smoke sensor in the cabin sub-area are both connected to the sensor data processing board via Wi-Fi, and transmit the CO concentration value and smoke concentration value at the detection point to the sensor data processing board via Wi-Fi.

[0054] 3. The edge processing layer analyzes and processes the infrared video data and sensor data obtained by the data acquisition layer, and sends the abnormal fire data and four fire risk scores based on the infrared video data and sensor data to the communication unit of the fire decision layer; the video data processing board analyzes and processes the infrared video data, that is, the video data processing board processes the temperature data and thermal imaging video data within the monitoring range of the infrared thermal imaging sensor; the sensor data processing board analyzes and processes the sensor data, that is, the sensor data processing board processes the gas concentration and smoke concentration data within the detection range of the sensor; when one of the video data processing board and the sensor data processing board first detects that the data it is responsible for processing is abnormal, it marks the area where the abnormal data is located and sends it to the other party via WiFi, while increasing its own sampling frequency and performing fire judgment scoring on the abnormal area.

[0055] For example, the video data processing board first detects abnormal temperature data: When the video data processing board first detects an anomaly in temperature data, it marks the location of the anomaly and increases the sampling frequency. Then, in subsequent thermal image frames, it performs grayscale and temperature measurements on the marked area, generating fire risk scores based on the thermal image frame and temperature matrix, respectively. Simultaneously, the video data processing board transmits the anomaly location information to the sensor data processing board. Upon receiving this information, the sensor data processing board increases the sampling frequency of the sensor data and generates fire risk scores based on the gas and smoke sensor data, respectively.

[0056] Fourth, the decision-making unit uses the fire data and fire risk scores sent by the edge processing layer to initiate corresponding linkage measures based on the fire level, and conducts BP neural network training to optimize the weight parameters of thermal imaging data, temperature data, CO gas concentration, and smoke concentration in the judgment of fire hazard level, thereby improving the accuracy of fire prior judgment and achieving rapid response.

[0057] 5. If the communication unit of the shore-based cloud system receives a distress signal and fire anomaly data, the fire anomaly data will be stored in the storage unit for backup to facilitate later accident tracing, and the alarm unit will notify the base station to arrange fire rescue for the ship.

[0058] Example 3 1. The video data processing board analyzes and processes infrared video data as follows: (1) The infrared thermal imaging video stream is divided into sequence frame images according to the timestamp, and each image frame contains temperature matrix information. Camera equipment at the time , , the image frame obtained is .

[0059] (2) Infrared thermal imaging pictures Convert to grayscale image, grayscale processing method uses weighted average method:

[0060] in, 、 、 Represents the red, green and blue components in position The brightness value, 、 、 Respectively represent the weights of the red, green, and blue components in grayscale calculation; considering that in fire detection tasks, the flame area is usually presented in red, yellow, and white in thermal imaging images. Therefore, in the grayscale process, the present invention assigns a higher weight to the color channel reflecting the high temperature area. The value can be 0.6, The value can be 0.3, The value can be 0.1.

[0061] (3) In order to unify the dynamic range of the grayscale image and facilitate threshold discrimination, the grayscale image Normalization is performed to standardize the image grayscale dynamic range to [0, 255]. The calculation formula is as follows:

[0062] in, 、 Respectively represent the minimum and maximum grayscale values ​​in the current image.

[0063] (4) Set the grayscale threshold , binarize the grayscale image, extract the high temperature candidate area, and obtain the flame mask image , which is defined as follows:

[0064] Using eight-neighborhood calculation method to image All the pixels with a value of 1 in the image are analyzed for connected domain extraction. connected regions, respectively , ,…, : .

[0065] (5) Set the minimum fire source area judgment threshold as , using the formula Calculate the area of ​​each connected region ,like Greater than or equal to , indicating that the current frame has identified a suspected fire area, and the suspected fire area is marked as a key focus area , and set the fire confirmation flag value of the frame Set to 1 to start the subsequent multi-frame tracking detection mechanism.

[0066] (6) Continuously read the following frames from the current frame Frame images, and perform the above (2)-(5) processing and analysis on each frame image, assuming that the continuous frame set is: , for each frame image in the frame set, repeatedly extract the flame mask image in the image , and record whether it is in the key focus area And whether the flame area meets the threshold condition, if there is a suspected fire area that meets the condition in the frame, then the fire confirmation flag value of the frame is set to:

[0067] Where n=1,2,…,N; Calculate the fire risk score for the current detection period , which is calculated as follows:

[0068] in: , indicating infrared camera equipment exist The confidence score of fire events at all times; the higher the score, the more stable and continuous high temperature anomalies are in the area, and the higher the probability of fire.

[0069] According to the above steps, the fire risk score based on the infrared thermal imaging image can be obtained. If any data anomaly occurs, the video data processing board will increase the sampling frequency and focus on the abnormal data area. , and perform grayscale threshold judgment according to the above steps to obtain the fire risk score of the area based on the thermal imaging image .

[0070] In the early stages of a fire, the local temperature often continues to rise slowly, but has not yet reached the stage of an open flame. Therefore, it is difficult to detect the early temperature rise of a fire based solely on image features. Infrared cameras output a two-dimensional temperature matrix during each frame of image acquisition, and each thermal image frame corresponds to a temperature matrix. Therefore, the present invention uses the temperature data captured by the infrared camera to construct a temperature rise trend model, enabling earlier fire warnings.

[0071] 2. The video data processing board analyzes and processes the temperature data as follows: (1) When there is no data anomaly, the video data processing board will first calculate the initial background temperature mean of each pixel by obtaining the temperature matrix of N consecutive frames. , and use this average as the normal temperature baseline for comparison with subsequent temperature rise values; secondly, obtain the temperature matrix in the new frame , using the formula Calculate the temperature difference between the new frame and the background, and determine whether there is a temperature rise. The judgment is based on: , is the temperature rise threshold; if the temperature rise condition is met, the eight-neighborhood calculation method is used to find the continuous temperature rise area , set the area threshold of the temperature rise area to If the area , it is considered that there is a suspected temperature rise area; then, the average temperature increment of the temperature rise area in the subsequent L frames is calculated: , and then use linear regression to calculate the slope , if the slope , indicating that the area is rapidly heating up, constituting an early fire warning; finally, combining the temperature rise amplitude, temperature rise rate, and temperature rise area parameters, a fire risk score based on temperature data is obtained. , the scoring criteria are as follows:

[0072] in, , , , The fire risk score is respectively the temperature rise amplitude, temperature rise rate and temperature rise area. Weight parameter in the calculation.

[0073] (2) When any fire data is abnormal, the video data processing board will increase the sampling frequency and use the abnormal data area as the temperature rise area. , follow the above steps to judge the temperature rise trend and calculate the fire risk score of the area based on the temperature matrix data .

[0074] 3. The sensor data processing board performs edge processing and analysis on CO concentration data and smoke concentration data as follows: (1) The CO gas sensor and smoke sensor array arranged in the cabin sub-area obtain the CO concentration and smoke concentration at each detection point, and the sampling period is a fixed interval , the concentration data collected in each sampling period is recorded as:

[0075] in, represents the concentration at the current moment, and n is the length of the historical time window of the concentration data.

[0076] (2) Set three concentration threshold points , and divide these three points into two continuous subintervals and , and are mapped to score intervals respectively and ; Set concentration score , concentration scoring function for:

[0077] in, , ; Represents the upper limit of safe concentration, Represents the starting point of the warning concentration, represents the upper limit of the map concentration; Represents the highest value of the current concentration score, Representative concentration reached Concentration score at the value; (3) Settings The time window step selected for analyzing the concentration growth trend is: For the current moment The concentration value of each sampling point, Indicates the concentration value n sampling periods ago, and the average growth rate of the concentration is calculated using the linear slope method :

[0078] And map the average growth rate of concentration to the concentration growth rate score , , mapping function is defined as:

[0079] in, The turning point of medium-speed growth is the warning threshold for rapid growth; represent The rating value at time (4) Score the current concentration Concentration growth trend score Linear weighting is performed to obtain a fire hazard score based on concentration :

[0080] in, , is the weight coefficient, satisfying , .

[0081] Following these steps, a fire risk score can be derived based on the CO and smoke concentrations in the cabin area. If any data anomalies occur, the sensor data processing board increases the sampling frequency and, following the aforementioned steps, determines the CO and smoke concentration thresholds and growth rates for the marked anomaly data area, deriving a fire risk score for that area based on these concentrations.

[0082] Example 4 like Figure 4 As shown in the figure, the decision-making unit in the fire decision layer uses a method that combines a priori scorer based on a BP neural network with a main model. This method enables the fire determination network to output the final fire level while simultaneously optimizing the weight parameters of the priori scorer. This improves the accuracy of the prior fire judgment and enables a rapid response of the fire decision system's coordinated measures.

[0083] The prior scorer calculates the total fire risk score based on the received thermal imaging data, temperature data, CO gas concentration, and smoke concentration fire risk score. , the prior scorer is calculated as:

[0084] in, ; 、 、 、 is the prior weight parameter to be trained, b is the bias term, σ is the Sigmoid activation function, Reflects the credibility of the current edge anomaly.

[0085] A BP neural network model containing four fire judgment features including thermal imaging image features, temperature vector, gas concentration vector and smoke concentration vector is constructed as the main fire judgment model to output the final fire hazard level. The present invention can truly reflect the reliability of the current edge node judgment. The joint loss function is constructed as follows:

[0086] in, is the classification loss of the main neural network, which is used to supervise the prediction accuracy of the final fire grade; is the loss term of the prior scorer, used for reverse training 、 、 、 ; The expected prior value set according to the true label is recorded as ,but .

[0087] The gradient descent optimization method with momentum is introduced in the training phase as follows:

[0088]

[0089] in, Represents trainable parameters 、 、 、 ; is the learning rate; is the momentum factor; is the joint loss function about gradient; is the weight update amount of the previous iteration.

[0090] When the prior score When the fire hazard score is close to the fire hazard score obtained by the main neural network based on the four-way original data, it means that the model has converged and the model can be put into the deployment stage. The present invention sets a priori threshold judgment mechanism in the deployment stage: when Exceeding the set threshold , directly send out fire alarm signals, thus achieving a more real-time fire warning function. Does not exceed the set threshold , then follow the above steps to carry out the process of prior + neural network. The scores are mapped to fire hazard levels, which are determined as follows:

[0091] Among them, the present invention sets according to actual needs =30, =50, =80. =1 means the danger level is 1, which means there is no open flame at present, but there is a fire risk. At this time, the control unit links the broadcast and alarm lights to issue a fire warning; =2 means the fire danger level is 2, indicating that an open flame has appeared. At this time, the control unit will link the broadcast and alarm lights to issue a fire alarm, and at the same time link the fire protection system to carry out targeted fire extinguishing. =3 represents the fire danger level of 3, indicating that the current fire is large and has a tendency to spread. At this time, the control unit links the broadcast and alarm lights to issue a fire alarm, links the fire protection system to carry out targeted fire extinguishing, and at the same time, the communication unit sends a fire distress signal and fire abnormality data to the shore-based cloud layer communication unit.

Claims

1. The multi-level fire alarm system for unmanned cabins in ships based on edge computing is characterized by: It includes an intelligent shipborne system and a shore-based cloud system connected via satellite communication. The intelligent shipborne system is deployed inside the ship's engine room and consists of a data acquisition layer, a data processing layer and a fire decision layer; the shore-based cloud system is a cloud of a shore-based system set on land, and consists of a communication unit, a storage unit and an alarm unit. The storage unit stores fire data transmitted by the fire decision layer, the communication unit communicates data with the fire decision layer, and the alarm unit receives a fire distress signal sent by the fire decision layer and sends it to the shore-based system to alarm.

2. The edge computing-based multi-level fire alarm system for unmanned ship cabins according to claim 1 is characterized in that: The data acquisition layer includes an infrared thermal imaging sensor, a CO gas sensor and a smoke sensor; the edge processing layer includes a video data processing board and a sensor data processing board. The infrared thermal imaging sensor transmits data with the video data processing board, and the CO gas sensor and the smoke sensor both transmit data with the sensor data processing board; the fire decision layer includes a communication unit, a decision unit, a control unit and a storage unit that are interconnected, transmits data with the edge processing layer and the shore-based cloud system, determines the fire hazard level, and links the cabin alarm and fire protection system. Finally, the data is stored locally.

3. The edge computing-based multi-level fire alarm system for unmanned ship cabins according to claim 2 is characterized in that: The video data processing board and the sensor data processing board both include a central processing and system memory unit, a wifi wireless communication unit, a wired communication interface, a universal serial port interface, and a power supply and reset module. The central processing and system memory unit includes a main control chip and an internal memory to complete system startup boot, file system loading, and edge AI model storage; the wifi wireless communication unit is an external Wi-Fi module, which establishes a wireless connection and communication with the data acquisition layer; the wired communication interface is a Gigabit Ethernet PHY chip, which is connected to the main control chip; the universal serial port interface is a serial port connected to the UART interface of the main control chip to complete debugging and module expansion connection; the power supply and reset module is an onboard DC-DC power supply, and an introduced watchdog protection circuit.

4. The method of using the edge computing-based multi-level fire alarm system for unmanned ship cabins according to claim 3 is characterized in that: The steps include: Step a: Divide the ship's engine room into smaller sub-areas based on internal structural layout, equipment density, heat source distribution, and ventilation paths, ensuring that each sub-area has spatial independence and fire risk homogeneity. In each sub-area, multiple infrared thermal imaging sensors, CO gas sensors, and smoke sensors are placed and connected to the edge processing layer. In step b, the infrared thermal imaging sensor in the cabin sub-area is connected to the video data processing board via Wi-Fi, and transmits the infrared video data, including the thermal imaging video data and the temperature information of the coverage area, to the video data processing board via Wi-Fi. The CO gas sensor and the smoke sensor in the cabin sub-area are also connected to the sensor data processing board via Wi-Fi, and transmit the CO concentration value and the smoke concentration value at the detection point to the sensor data processing board via Wi-Fi. Step c: The edge processing layer analyzes and processes the infrared video data and sensor data acquired by the data acquisition layer, and sends abnormal fire data and fire risk scores based on the infrared video data and sensor data to the communication unit of the fire decision layer; In step d, the decision-making unit uses the fire data and fire risk scores sent by the edge processing layer to initiate corresponding linkage measures based on the fire level. It also conducts BP neural network training to optimize the weight parameters of thermal imaging data, temperature data, CO gas concentration, and smoke concentration in determining the fire hazard level, thereby improving the accuracy of fire prior judgment and achieving rapid response. In step e, if the communication unit of the shore-based cloud system receives the distress signal and fire abnormality data, the fire abnormality data is stored in the storage unit for backup to facilitate later accident tracing, and the alarm unit notifies the base station to arrange fire rescue for the ship.

5. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 4 is characterized in that: Step c includes the following specific contents: The video data processing board analyzes and processes the infrared video data, that is, the video data processing board processes the temperature data and thermal imaging video data within the monitoring range of the infrared thermal imaging sensor; the sensor data processing board analyzes and processes the sensor data, that is, the sensor data processing board processes the gas concentration and smoke concentration data within the sensor detection range; when one of the video data processing board and the sensor data processing board first detects that the data it is responsible for processing is abnormal, it marks the area where the abnormal data is located and sends it to the other party via WiFi, while increasing its own sampling frequency and performing a fire judgment score on the abnormal area.

6. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 4 or 5, characterized in that: The BP neural network training described in step d uses a method combining the prior scorer of the BP neural network with the main model to reversely optimize the weight parameters of the prior scorer, specifically including the following: The prior scorer calculates the total fire risk score based on the received thermal imaging data, temperature data, CO gas concentration, and smoke concentration fire risk score. , the prior scorer is calculated as: in, ; 、 、 、 is the prior weight parameter to be trained, b is the bias term, σ is the Sigmoid activation function, Reflects the credibility of the current edge anomaly; A BP neural network model containing four fire judgment features including thermal imaging image features, temperature vector, gas concentration vector and smoke concentration vector is constructed as the main fire judgment model to output the final fire hazard level. The present invention can truly reflect the reliability of the current edge node judgment. The joint loss function is constructed as follows: in, is the classification loss of the main neural network, which is used to supervise the prediction accuracy of the final fire grade; is the loss term of the prior scorer, used for reverse training 、 、 、 ; The expected prior value set according to the true label is recorded as ,but ; The gradient descent optimization method with momentum is introduced in the training phase as follows: in, Represents trainable parameters 、 、 、 ; is the learning rate; is the momentum factor; is the joint loss function about gradient; is the weight update amount of the previous iteration.

7. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 5 is characterized in that: The video data processing board analyzes and processes the infrared video data as follows: The infrared thermal imaging video stream is divided into sequence frame images according to the timestamp, and each image frame contains temperature matrix information. Camera equipment at the time , , the image frame obtained is ; Infrared thermal imaging pictures Convert to grayscale image, grayscale processing method uses weighted average method: in, 、 、 Represents the red, green and blue components in position The brightness value, 、 、 Represent the weights of the red, green and blue components in grayscale calculation respectively; In order to unify the dynamic range of the grayscale image and facilitate threshold discrimination, the grayscale image Normalization is performed to standardize the image grayscale dynamic range to [0, 255]. The calculation formula is as follows: in, 、 Respectively represent the minimum and maximum grayscale values ​​in the current image; Set grayscale threshold , binarize the grayscale image, extract the high temperature candidate area, and obtain the flame mask image , which is defined as follows: Using eight-neighborhood calculation method to image All the pixels with a value of 1 are analyzed for connected domain extraction. Suppose that a total of connected regions, respectively , ,…, : ； Set the minimum fire source area judgment threshold to , using the formula Calculate the area of ​​each connected region ,like Greater than or equal to , indicating that the current frame has identified a suspected fire area, and the suspected fire area is marked as a key focus area , and set the fire confirmation flag value of the frame Set to 1 to start the subsequent multi-frame tracking detection mechanism; Continuously read the following frames from the current frame Frame images, and perform the above processing and analysis on each frame image, assuming that the continuous frame set is: , for each frame image in the frame set, repeatedly extract the flame mask image in the image , and record whether it is in the key focus area And the flame area meets the threshold condition. If there is a suspected fire area that meets the condition in the frame, the fire confirmation flag value of the frame is set to: Where n=1,2,…,N; Calculate the fire risk score for the current detection period , which is calculated as follows: in: , indicating infrared camera equipment exist The confidence score of fire events at all times; the higher the score, the more stable and continuous high temperature anomalies are in the area, and the higher the probability of fire.

8. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 7 is characterized in that: The video data processing board analyzes and processes the temperature data as follows: When there is no data anomaly, the video data processing board will first calculate the initial background temperature mean of each pixel by obtaining the temperature matrix of N consecutive frames. , and use this average as the normal temperature baseline for comparison with subsequent temperature rise values; secondly, obtain the temperature matrix in the new frame , using the formula Calculate the temperature difference between the new frame and the background, and determine whether there is a temperature rise. The judgment is based on: , is the temperature rise threshold; if the temperature rise condition is met, the eight-neighborhood calculation method is used to find the continuous temperature rise area , set the area threshold of the temperature rise area to If the area , it is considered that there is a suspected temperature rise area; then, the average temperature increment of the temperature rise area in the subsequent L frames is calculated: , and then use linear regression to calculate the slope , if the slope , indicating that the area is rapidly heating up, constituting an early warning of fire; finally, combining the temperature rise amplitude, temperature rise rate, and temperature rise area parameters, a fire risk score based on temperature data is obtained. , the scoring criteria are as follows: in, , , , The fire risk score is respectively the temperature rise amplitude, temperature rise rate and temperature rise area. Weight parameter in the calculation.

9. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 7 or 8, characterized in that: The sensor data processing board performs edge processing and analysis on CO concentration data and smoke concentration data as follows: The CO gas sensor and smoke sensor array arranged in the cabin sub-area obtain the CO concentration and smoke concentration at each detection point. The sampling period is a fixed interval. , the concentration data collected in each sampling period is recorded as: in, represents the concentration at the current moment, and n is the length of the historical time window of the concentration data; Set three concentration threshold points , and divide these three points into two continuous subintervals and , and are mapped to score intervals respectively and ; Set concentration score , concentration scoring function for: in, , ; Represents the upper limit of safe concentration, Represents the starting point of the warning concentration, represents the upper limit of the map concentration; Represents the highest value of the current concentration score, Representative concentration reached Concentration score at the value; set up The time window step selected for analyzing the concentration growth trend is: For the current moment The concentration value of each sampling point, Represents the concentration value n sampling periods ago, and the average growth rate of the concentration is calculated using the linear slope method : And map the average growth rate of concentration to the concentration growth rate score , , mapping function is defined as: in, The turning point of medium-speed growth is the warning threshold for rapid growth; represent The rating value at time The current concentration score Concentration growth trend score Linear weighting is performed to obtain a fire hazard score based on concentration : in, , is the weight coefficient, satisfying , .

10. The multi-level fire alarm method for unmanned ship cabin based on edge computing according to claim 6 is characterized in that: When the BP neural network calculates the fire hazard level, The scores are mapped to fire hazard levels: =1 means the danger level is 1, which means there is no open flame at present, but there is a fire risk. At this time, the control unit links the broadcast and alarm lights to issue a fire warning; =2 means the fire danger level is 2, indicating that an open flame has appeared. At this time, the control unit will link the broadcast and alarm lights to issue a fire alarm, and at the same time link the fire protection system to carry out targeted fire extinguishing. =3 represents the fire danger level of 3, indicating that the current fire is large and has a tendency to spread. At this time, the control unit links the broadcast and alarm lights to issue a fire alarm, links the fire protection system to carry out targeted fire extinguishing, and at the same time, the communication unit sends a fire distress signal and fire abnormality data to the shore-based cloud layer communication unit.

Images