Ship fire detection method, device and equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a ship fire detection method, device and equipment and a storage medium, and the method comprises the steps: constructing a ship fire detection network LFADF-Net, and inputting a ship fire training data set into the ship fire detection network LFADF-Net; according to the ship fire training data set, parameters in the ship fire detection LFADF-Net network are optimized and trained through a pixel-based loss function, and a trained LFADF-Net ship fire detection network model is obtained; collecting video data of a target ship; fire feature extraction and ship fire detection are performed on the video data of the target ship through the trained LFADF-Net ship fire detection network model, a ship fire detection result is obtained, the calculation efficiency of the LFADF-Net ship fire detection network model is high, and the fire of the ship in the whole stage can be accurately detected in real time.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a ship fire detection method, device, equipment and storage medium. Background Art

[0002] With the rapid development of the shipbuilding industry system, it has been widely used in the fields of maritime transportation, military defense and resource development. However, in the process of rapid development of the shipbuilding industry, fires on ships continue to occur. Due to the complex internal space of ships, dense electrical equipment, and a large amount of fuel and other flammable substances, fire has become a major safety hazard for ships. Once a fire occurs inside a ship, it may not only cause huge losses of equipment and property, but also threaten the lives of crew members and even seriously pollute the marine ecological environment. With the rapid development of ports and shipping, the number of ships invested is also gradually increasing. Therefore, it is of great significance to design an algorithm that can quickly and accurately detect early fires. The sooner the fire is warned, the less loss of life and property will be caused. In addition, the rapid and accurate detection of early fires can also reserve more time for subsequent firefighting decisions, which is of great significance to improving the ship damage control system.

[0003] Traditional fire detection technology uses sensors to detect fires. During a fire, a large amount of heat is released, which in turn raises the temperature of the air around the flame, accompanied by smoke and light. Most of the sensors currently designed detect the characteristics of the above burning materials. Their working principle is to use the particles generated by the fire to impact the sensor. When the detected particle parameters reach the alarm threshold, the alarm system will be triggered to alarm. According to the different characteristics of the particles detected by the sensor, fire sensors can be divided into smoke fire sensors, temperature fire sensors, light fire sensors and composite fire sensors. Using sensors to detect fires has the advantages of simple structure and stable performance, and is more suitable for indoor areas in small spaces. However, due to the limitations of their working principles, these sensor-based fire detection systems have obvious disadvantages: they are not suitable for large areas, because the sensor can only detect fires through particles generated by the fire when it is close to the fire source. At a long distance, the particle propagation speed is limited, which will cause the sensor to alarm with a delay. At the same time, in the process of particle propagation, the particle characteristics gradually weaken as the distance increases, so that the sensor cannot detect the fire. Therefore, sensors are not applicable in the large space of ships. In addition, these sensors cannot provide detailed on-site information such as the initial location of the fire, the size of the fire, the direction of smoke propagation, and the growth rate, and their contribution to assisting fire-fighting decisions is limited.

[0004] With the popularity of monitoring systems in recent years, the commonly used methods for detecting fires can be detected by video monitoring systems in addition to using sensor detection systems. At present, a large number of researchers have verified that video-based fire detection methods can effectively overcome the limitations of sensors. Compared with sensor detection, video-based fire detection has the characteristics of large detection range, fast response speed, strong anti-interference ability and low cost, and can more intuitively feedback detailed information on the fire scene. Therefore, most scenes have begun to use video monitoring to detect fires. From the perspective of algorithms, video-based fire detection methods can be divided into two types: traditional machine learning-based methods and deep learning-based methods. Traditional machine learning-based fire detection methods mainly rely on manually extracting flame and smoke features and designing classifiers based on these features. Among them, the features of flames and smoke are divided into static features such as color and texture shape, and dynamic features such as area growth, morphological changes, edge changes, and moving directions. After manually extracting features, they need to be combined with corresponding classification algorithms to determine whether a fire occurs in the video area. However, the process of manually extracting image features is cumbersome, complex, and requires a lot of professional experience, and is highly subjective. Once the manually extracted features are biased, the accuracy of the entire model will be affected. Moreover, the manually extracted image features can only be used for specific data sets and have poor generalization capabilities for fire images in different scenarios, resulting in low model detection accuracy.

[0005] Compared with traditional machine learning methods, fire detection methods based on deep learning have the advantages of fast detection speed, high accuracy, and strong model generalization ability. First, deep learning networks can automatically extract features from large amounts of data, avoiding the complexity and limitations of manual feature extraction. Secondly, deep learning models perform better when processing high-dimensional image data and can effectively improve the accuracy of detection. In addition, deep learning methods have strong generalization and robustness and can adapt to different fire scenes and environments. At present, the shipbuilding industry is gradually developing towards intelligence, automation, and large-scale development. Important places on ships are equipped with surveillance cameras. Therefore, as long as the number of deep learning model parameters is limited to a certain range, it can be successfully deployed on the ship monitoring system. Therefore, after comprehensively considering the above three fire detection methods and ship scenarios, it can be seen that the use of deep learning methods to detect ship fires has obvious advantages.

[0006] Although the deep learning method has performed well in many aspects in detecting ship fires, it also has some limitations. First, there is a lack of fire datasets that meet the ship scene. Most of the images in the current public fire datasets are collected in forests, and it is difficult to obtain fire data related to ships. This has led to the current fire detection still staying in forest scenes, and the fire detection model for ship scenes has weak generalization ability. Secondly, there is a lack of early fire data in the current fire datasets. In order to extinguish fires more quickly, the fire detection system should be able to alarm in the early stages of the fire. However, most of the current public fire datasets are in the complete combustion stage. This is because the fires in the public datasets are all emergencies recorded by humans in life. Early fires in daily life are difficult to notice, so there is a lack of early fire data. In addition, it is difficult for the fire detection system to take into account the real-time and accuracy of the model. The number of parameters of deep learning models with high detection accuracy is often large, making it difficult to deploy them on the monitoring system of ships. Models with smaller parameters are prone to false alarms and missed alarms. Therefore, it is difficult to take into account the accuracy and speed of the model.

[0007] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0008] The main purpose of the present invention is to provide a ship fire detection method, device, equipment and storage medium, aiming to solve the technical problem that the accuracy and efficiency of fire detection in the prior art cannot be taken into account at the same time.

[0009] To achieve the above object, the present invention provides a ship fire detection method, the ship fire detection method comprising: Constructing a ship fire detection network LFADF-Net, and inputting a ship fire training data set into the ship fire detection LFADF-Net network; According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model; Collect video data of target ships; Fire feature extraction and ship fire detection are performed on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

[0010] Preferably, before the step of constructing a ship fire detection network LFADF-Net and inputting a ship fire training data set into the ship fire detection LFADF-Net network, the step further includes: Collect video data of various ship fires; Processing the ship fire video data to obtain ship fire image data, performing data enhancement on the ship fire image data, and constructing a ship fire annotation data set; The ship fire annotated dataset is annotated to obtain a ship fire training dataset.

[0011] Preferably, the performing fire feature extraction and ship fire detection on the video data of the target ship by using the trained LFADF-Net ship fire detection network model to obtain the ship fire detection result includes: Using the trained LFADF-Net ship fire detection network model, fire feature extraction and ship fire detection are performed on the video data of the target ship, and the location of the fire and smoke and the corresponding confidence threshold are output; Determining whether there is a fire on the target ship according to the location of the fire and smoke and the corresponding confidence threshold; If it is determined that there is a fire, the video data of the target ship is sent to the temporal optical flow model, and the direction and propagation speed of the fire are calculated according to the movement information of the fire and smoke between the frame images in the video data of the target ship; If it is determined that there is no fire, frame sampling detection is performed based on the image input of the differential frame in the video data of the target ship to obtain a fire optical flow estimation result. Based on the fire optical flow estimation result, a fire or smoke optical flow feature vector is obtained to check whether there is a suspicious fire on the target ship.

[0012] Preferably, the ship fire detection network LFADF-Net comprises: a lightweight feature extraction backbone, an efficient feature fusion module and a yolov8 detection head.

[0013] Preferably, the lightweight feature extraction backbone includes a GBS module and a GSME module; the efficient feature fusion module includes a DySample module, a GTME module and a GSDI module; Eliminate redundant information in the ship fire training data set through the GBS module to obtain de-redundant video data; The GSME module is used to compensate the de-redundant video data and extract multi-scale features to obtain a feature map; Adaptively reducing background information in the feature map based on a dynamic range factor by the DySample module; Through the GTME module, the text attention mechanism is used to dynamically guide the image to focus on the irregular area of ​​the fire sample through the text weight; Through the GSDI module, spatial and channel attention mechanisms are applied to capture global channel information and local spatial details respectively, downsample large feature maps to obtain low-level position information, upsample small feature maps to obtain high-level semantic information, and integrate all features; The yolov8 detection head receives the input of the feature map and detects the fire location and confidence.

[0014] Preferably, according to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model, including: According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained through the mixed loss function obtained by rectangular box loss, object loss, classification loss and weighted classification loss to obtain a trained LFADF-Net ship fire detection network model.

[0015] Preferably, after performing fire feature extraction and ship fire detection on the video data of the target ship through the trained LFADF-Net ship fire detection network model and obtaining the ship fire detection result, the method further comprises: According to the ship fire detection result, the alarm level is adjusted, and an alarm is issued according to the alarm level.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also proposes a ship fire detection device comprising: A construction module is used to construct a ship fire detection network LFADF-Net, and input the ship fire training data set into the ship fire detection LFADF-Net network; A training module, used for optimizing and training the parameters in the ship fire detection LFADF-Net network according to the ship fire training data set by means of a pixel-based loss function, so as to obtain a trained LFADF-Net ship fire detection network model; A collection module, used to collect video data of the target ship; The detection module is used to extract fire features and detect ship fires on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

[0017] In addition, to achieve the above-mentioned purpose, the present invention also proposes a ship fire detection device, on which a ship fire detection program is stored, and when the ship fire detection program is executed by a processor, the steps of the ship fire detection method described above are implemented.

[0018] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a ship fire detection program is stored, and when the ship fire detection program is executed by a processor, the steps of the ship fire detection method described above are implemented.

[0019] In the present invention, by constructing a ship fire detection network LFADF-Net, a ship fire training data set is input into the ship fire detection LFADF-Net network; according to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model; the video data of the target ship is collected; the video data of the target ship is subjected to fire feature extraction and ship fire detection by the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result. The ship fire detection LFADF-Net network has high computational efficiency and can detect fires in the entire stage of a ship in real time and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural schematic diagram of a ship fire detection method in a hardware operating environment involved in an embodiment of the present invention; Figure 2 This is a flow chart of a first embodiment of a ship fire detection method according to the present invention; Figure 3 This is the overall architecture diagram of LFADF-Net in the embodiment of the ship fire detection method of the present invention; Figure 4 It is a specific structural diagram of GhostConv in the embodiment of the ship fire detection method of the present invention; Figure 5 Figure 1 is a structural and detailed diagram of the GSME in the embodiment of the ship fire detection method of the present invention, wherein (a) is the overall architecture of the GSME, (b) is a Dark Ghost-Bottleneck configuration formed by stacking multiple Ghost Bottlenecks, and (c) is the structure of the Ghost Bottleneck; Figure 6 This is a specific structural diagram of the ADFF module in the embodiment of the ship fire detection method of the present invention; Figure 7 This is a specific structural diagram of DySample in the embodiment of the ship fire detection method of the present invention; Figure 8 This is a specific structural diagram of the GTME module in the embodiment of the ship fire detection method of the present invention; Fig. 9 This is a specific structural diagram of GSDI in an embodiment of the ship fire detection method of the present invention; Fig.10 This is a flow chart of model format conversion in an embodiment of the ship fire detection method of the present invention; Fig.11 A diagram showing the specific steps of the ship fire detection method and fire motion information calculation in an embodiment of the ship fire detection method of the present invention; Fig.12 This is a diagram showing the architecture of a rapid ship fire detection system in an embodiment of a ship fire detection method of the present invention; Fig.13 This is a structural block diagram of the first embodiment of the ship fire detection device of the present invention.

[0021] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0023] Reference Figure 1 , Figure 1 The schematic diagram of the structure of ship fire detection equipment in the hardware operating environment involved in the embodiment of the present invention is shown in FIG.

[0024] like Figure 1 As shown, the ship fire detection equipment may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), and the user interface 1003 may also include a standard wired interface and a wireless interface. The wired interface of the user interface 1003 may be a USB interface in the present invention. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable memory (Non-volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0025] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the ship fire detection equipment, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0026] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a ship fire detection program.

[0027] exist Figure 1 In the ship fire detection device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the user device; the ship fire detection device calls the ship fire detection program stored in the memory 1005 through the processor 1001, and executes the ship fire detection method provided by the embodiment of the present invention.

[0028] Based on the above hardware structure, an embodiment of the ship fire detection method of the present invention is proposed.

[0029] Reference Figure 2 , Figure 2 The figure is a flow chart of the first embodiment of the ship fire detection method of the present invention, and the first embodiment of the ship fire detection method of the present invention is proposed.

[0030] In a first embodiment, the ship fire detection method comprises the following steps: Step S10: construct a ship fire detection network LFADF-Net, and input the ship fire training data set into the ship fire detection LFADF-Net network.

[0031] In a specific implementation, the execution subject of this embodiment is the ship fire detection device, wherein the ship fire detection device can be a personal computer, a lower computer stm32f103c8t6 or a server and other electronic devices, and this embodiment does not limit this. Construct a ship fire detection network LFADF-Net, and input the ship fire training data set into the ship fire detection LFADF-Net network. The structure of LFADF-Net is as follows Figure 3 As shown in the figure, it mainly consists of three parts, including lightweight feature extraction (LMFE) backbone, efficient feature fusion module (ADFF module) and yolov8 detection head. The model has high computational efficiency and can detect fires in the entire stage of a ship in real time and accurately. In addition, this embodiment creates a fire dataset that is more in line with the ship scenario, which covers the three stages of fire, complete combustion and extinguishing.

[0032] Furthermore, in this embodiment, the ship fire detection network LFADF-Net includes: a lightweight feature extraction backbone, an efficient feature fusion module and a yolov8 detection head.

[0033] Further, in this embodiment, the lightweight feature extraction backbone includes a GBS module and a GSME module; the efficient feature fusion module includes a DySample module, a GTME module and a GSDI module; Eliminate redundant information in the ship fire training data set through the GBS module to obtain de-redundant video data; The GSME module is used to compensate the de-redundant video data and extract multi-scale features to obtain a feature map; Adaptively reducing background information in the feature map based on a dynamic range factor by the DySample module; Through the GTME module, the text attention mechanism is used to dynamically guide the image to focus on the irregular area of ​​the fire sample through the text weight; Through the GSDI module, spatial and channel attention mechanisms are applied to capture global channel information and local spatial details respectively, downsample large feature maps to obtain low-level position information, upsample small feature maps to obtain high-level semantic information, and integrate all features; The yolov8 detection head receives the input of the feature map and detects the fire location and confidence.

[0034] (1) Lightweight Feature Extraction Backbone In fire detection, the irregular shapes of flames and smoke result in a large amount of background information in the target box, which makes the feature extraction in the Yolov8s backbone network redundant and reduces the computational efficiency. To solve this problem, a LMFE backbone network consisting of GBS and GSME is designed to optimize resource utilization through linear operations while improving information integration. GBS is used to eliminate unnecessary information and reduce redundancy, while GSME uses the SimAM attention mechanism to mitigate the accuracy loss caused by information loss and enhance feature utilization. The structure and hyperparameters of the proposed LMFE backbone network are given in Table 1 below, where "Output-size" represents the feature map dimension (channel, width, height), and "F1"-"F6" represent the feature map after applying a specific operator.

[0035] Table 1 LMFE specific structure

[0036] The GBS module includes GhostConv, BN (Batch Normalization) and activation function SiLU. The calculation process is as follows: (1.1) The structure diagram of GhostConv is Figure 4 It performs 1×1 convolution to compress the input feature map in space and channel scale, and uses 5×5 convolution to restore the spatial information lost in the previous step. This design greatly reduces the operation cost while minimizing the loss of accuracy. The calculation process of GhostConv is as follows: (1.2) in, is the input feature map, is the final output feature map, Indicates the main features, Represents the ghost features of the output.

[0037] The GSME module aims to improve the accuracy by compensating for the reduction of GBS and achieve multi-scale feature extraction. Figure 5 As shown, GSME consists of GBS, Dark GhostBottleneck and SimAM. By synergistically utilizing GhostConv and SimAM, it achieves improved information utilization with fewer parameters while maintaining high accuracy.

[0038] The lightweight attention mechanism SimAM is integrated into the feature extraction process to improve accuracy and optimize feature utilization efficiency. Specifically, SimAM is applied to the feature map of the last layer of the GhostBottleneck module. It selectively amplifies or suppresses the input features through the interaction of excitation and inhibition between neurons, thereby improving efficiency. This process is completed by adjusting the weights of the feature map based on the calculated mean and variance. The calculation process of SimAM is as follows: (1.3) (1.4) (1.5) (1.6) in, represents the set of input feature maps, , and Represents the number of channels, height and width respectively. yes A single element in . and They represent the mean and variance of the feature map elements respectively. represents the total number of elements in the feature map, represents the attention weight, is a small constant used to prevent division by zero. Finally, Represents the output feature map.

[0039] (2) Adaptive feature fusion network The irregular shapes of flames and smoke not only provide a lot of background information within the bounding box, but also hinder the efficiency of traditional convolution in extracting fire-related features. Traditional convolution has difficulty in processing irregular samples due to its fixed shape. Furthermore, the use of many standard convolutions in the baseline model limits its ability to effectively extract and integrate features from irregular areas. To address such challenges, the ADFF module integrates three adaptive dynamic feature fusion methods (DySample, GTME, and GSDI) through a novel path. This design can realize the dynamic extraction of irregular fire samples, strengthen the integration of features at different scales, reduce parameters and computation, and thus improve the efficiency of multi-scale feature fusion.

[0040] The detailed structure of the ADFF module is as follows Figure 6 As shown in the figure, the feature maps F3 and F6 connected to the DySample module are extracted by the backbone network and concatenated. Subsequently, the concatenated feature maps are extracted by the GTME module to generate the feature map M1. This fusion path simultaneously obtains the high-resolution position information and high-dimensional semantic information of multi-scale fires. Next, the feature map M1 after dimensionality enhancement is fused with the low-level feature map F1 from LMEF through GTME to extract high-dimensional semantic information and generate the feature map P3. This operation enhances the small-scale semantic information of P3, which is helpful for the detection of small-scale fires. Then, the features of M1 are enhanced by the channel and spatial attention mechanism of the GSDI module and adaptively fused with the features of P4. At the same time, the same operation is performed on the enhanced M1 features, P3 and P5 feature maps. When M1 is fused with P4, its feature dimension remains unchanged, indicating that P4 has a good balance in position and semantic information and is suitable for detecting medium-scale targets. In contrast, when M1 is fused with P5, its feature dimension is reduced, which enhances the representation of high-resolution position details. At the same time, the semantic information of P3 is integrated into P5. The integration of the location information of M1 and the semantic information of P3 improves the applicability of P5 for large target fire detection. Finally, the detection head receives inputs from P3, P4, and P5 for fire location and confidence detection at different scales.

[0041] Dynamic convolution technology is often used to deal with irregular sample problems, but it will increase the computational cost and reduce efficiency. To solve this problem, this embodiment proposes a DySample point resampling method, which can effectively save computing resources. Figure 7 The structure of DySample is shown. This method uses a learnable parameter called the dynamic range factor to adaptively change the upsampled area and reduce the background information in the upsampled feature map. In addition, point resampling avoids the large computational burden and time requirements associated with dynamic convolution.

[0042] In addition, this implementation introduces Figure 8 The GTME module shown in Figure 1 is used to extract deeper features from multi-scale fire samples. The GTME module uses the text attention mechanism TextAttn to dynamically guide the image to focus on the irregular areas of the fire sample through text-based weights. The GBS and Dark GhostBottleneck structures in the GTME module are the same as those in the GSME module.

[0043] TextAttn fuses learnable text vectors with multi-scale fire feature maps to guide feature learning. As training progresses, these trained text weights become more focused on fire areas, helping adaptive attention. The process of TextAttn is as follows: First, a randomly initialized text vector is mapped to fit the format of the feature map. Next, this vector is multiplied with the feature map to calculate the position-specific weights. Note that the text vectors are trainable. The maximum value is then extracted from these weights and refined by transposing and applying the sigmoid function to obtain the final weights. Finally, the output feature map is generated by multiplying with these refined weights.

[0044] The calculation process of the text attention mechanism can be summarized as follows: (1.7) in, Represents the input feature map. Represents the output feature map obtained after multiplying the text weight. Represents the feature map obtained from text mapping. represents the applied sigmoid function, represents the position of the eigenvector.

[0045] GSDI module such as Fig. 9 As shown in Figure 2, feature maps are enhanced by integrating multi-level features to improve semantic and location information. First, spatial and channel attention mechanisms are applied ( and ) to capture global channel information and local spatial details respectively. Then, a custom IdentityMap is used ( ) performs adaptive sampling, downsampling large feature maps to obtain low-level location information, and upsampling small feature maps to obtain high-level semantic information. Finally, GhostConv integrates all features. This method provides more semantic and detailed location information for the features of small-scale fires, achieving a more comprehensive fusion of fire information.

[0046] The calculation process of GSDI is as follows: (1.8) (1.9) (1.10) In the formula, represents the i-th feature map of layer 0, and the same is true for other layers. It is worth noting that Represents the second feature map of layer 2, assumed to be a standard feature map (Identity). and They represent the spatial attention weight and channel attention weight of the i-th layer respectively. and Represent the height and width of the feature map respectively. Represents GhostConv.

[0047] Step S20: According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model.

[0048] It is understandable that the parameters in the LFADF-Net network are optimized and trained by the pixel-based loss function to obtain the trained LFADF-Net ship fire detection network model and obtain the optimal weight file Best.pt. Specifically, the parameters in the LEYolov8 network are optimized and trained by the mixed loss function (Loss) obtained by weighting the rectangular box loss (lossrect), the object loss (lossobj) and the classification loss (lossclc), to obtain the trained LEYolov8 ship fire detection network model. In this way, the network model can continuously adjust its parameters during the training process to achieve higher detection accuracy and lower false detection rate. The quality of the LEYolov8 network detection results can be measured by three indicators: rectangular box, confidence and classification probability. The rectangular box represents the size of the object and its exact location; the confidence represents the confidence of the predicted rectangle, ranging from 0 to 1, and the higher the value, the greater the possibility of finding the object in the rectangle; the classification probability represents the category of the object. Therefore, in order to enable the network to achieve better detection results, it is necessary to gradually optimize the above three indicators: rectangular box, confidence and classification probability. Therefore, the network's loss function needs to measure the degree of deviation between the predicted results and the expected labels in these three aspects. Therefore, the total loss function should include three parts of loss, namely rectangular box loss (lossrect), object loss (lossobj), and classification loss (lossclc). Therefore, the total loss of LEYolov8 is the weighted sum of these three losses, with the weights set to a=0.75, b=0.05, and c=0.3. . Both the confidence loss and classification loss are measured using the binary cross entropy loss function (BCEWithLogitsLoss), and the rectangular box loss is measured using CIoUloss. CIoUloss is based on IoUloss and additionally considers the distance between the predicted target box and the true label box, the overlap ratio, the aspect ratio of the box, and the penalty term to optimize the regression accuracy and speed of the rectangular box, which can make the predicted rectangular box regression more stable. The IoUloss and CIoUloss losses are shown in the following formulas:

[0049] In the above formula, IoU is defined as the ratio of the intersection and union between the predicted box A and the actual label box B. represents the Euclidean distance between the center point of the predicted box and the center point of the true box. c is the diagonal distance of the smallest enclosing rectangle containing the true box and the predicted box. and Defined by the above formula. wgt represents the width of the ground-truth box. hgt represents the height of the real box. Similarly, w represents the width of the predicted box. h represents the height of the predicted box.

[0050] After multiple rounds of iterative training, the trained LEYolov8 ship fire detection network model is obtained. The model can accurately detect fire and smoke areas in a given input image, and output the location coordinates of the fire and the fire or smoke category. In order to facilitate subsequent use and deployment, it is necessary to export the optimal weight file obtained by the model. This file contains all the knowledge learned by the model during the training process and can be used for real-time ship fire detection tasks.

[0051] Furthermore, in this embodiment, the step S20 includes: According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained through the mixed loss function obtained by rectangular box loss, object loss, classification loss and weighted classification loss to obtain a trained LFADF-Net ship fire detection network model.

[0052] Step S30: Collecting video data of the target ship.

[0053] It should be understood that to identify whether there is a fire on the target ship, the surveillance camera can be used to collect video data of the target area where the target ship is located, and send it to the control motherboard Nvidia JetsonTX2. The surveillance cameras of different scenes are used to collect real-time scene video data. The collected video data will be transmitted to the control motherboard in real time for processing.

[0054] Step S40: performing fire feature extraction and ship fire detection on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

[0055] It should be noted that the above-trained LFADF-Net ship fire detection network model can be deployed to the control motherboard Nvidia Jetson TX2. In this embodiment, the control motherboard adopts the Nvidia Jetson TX2 control motherboard, and the lower computer adopts the STM32 lower computer. After completing the training of the LFADF-Net ship fire detection network model, it is deployed to the control motherboard to realize real-time fire detection function. In order to realize real-time detection of campus fires or ship fires, the model is deployed to the JetsonTX2 embedded platform by combining TensorRT and Cuda acceleration libraries according to the scene characteristics of campus fires or ship fires, and the actual scene application of the model is completed. This process involves converting the optimal weight file of the model into a format suitable for the control motherboard, and ensuring that the input and output interfaces of the model are compatible with the hardware facilities of the motherboard. First, the model is converted to convert its weight file into a format recognizable by the control motherboard. At the same time, the structure of the model is adjusted to adapt to the processing power and memory limitations of the control motherboard. In this process, it is necessary to maintain the core functions and detection performance of the model to ensure that its operation on the control motherboard can meet the needs of actual applications. The process of converting the optimal weight file into a format suitable for the control motherboard is as follows Fig.10 shown.

[0056] Furthermore, in this embodiment, the step S40 includes: Using the trained LFADF-Net ship fire detection network model, fire feature extraction and ship fire detection are performed on the video data of the target ship, and the location of the fire and smoke and the corresponding confidence threshold are output; Determining whether there is a fire on the target ship according to the location of the fire and smoke and the corresponding confidence threshold; If it is determined that there is a fire, the video data of the target ship is sent to the temporal optical flow model, and the direction and propagation speed of the fire are calculated according to the movement information of the fire and smoke between the frame images in the video data of the target ship; If it is determined that there is no fire, frame sampling detection is performed based on the image input of the differential frame in the video data of the target ship to obtain a fire optical flow estimation result. Based on the fire optical flow estimation result, a fire or smoke optical flow feature vector is obtained to check whether there is a suspicious fire on the target ship.

[0057] In the specific implementation, the control mainboard extracts fire and smoke features and detects ship fires from the image data contained in the video data, calculates the fire direction and propagation speed using the time series optical flow fire motion calculation module, and finally sends the ship fire detection and calculation results to the lower computer stm32f103c8t6. Specifically, Fig.11 As shown, the positions of fire and smoke output by the LFADF-Net model and the corresponding confidence thresholds are obtained to determine whether there is a fire in the propagation. If there is no fire, it is determined whether there is a suspicious ship fire. The temporal optical flow estimation module in Nvidia Jetson TX2 is used to perform frame detection based on the image input of the differential frame to obtain the fire optical flow estimation result, which includes the horizontal displacement of fire or smoke pixels, the vertical displacement of fire or smoke pixels, the movement direction of fire or smoke pixels, and the speed of fire or smoke pixels. Then, the optical flow estimation result is filtered to remove noise and outliers to obtain the fire or smoke optical flow smoothing data. Finally, the fire or smoke optical flow smoothing data is normalized to obtain the fire or smoke optical flow feature vector. If there is a fire, the video frame images from the same monitoring video data are sent to the temporal optical flow model, and the direction and propagation speed of the fire are calculated based on the movement information of the fire and smoke between the acquired frame images.

[0058] In this embodiment, fire poses a major threat to the safe operation of ships, and it is crucial to achieve real-time fire detection. However, due to the high diversity of fire scenes, complex shapes of flames and smoke, and redundant model parameters, there are still many challenges in improving the accuracy and real-time performance of ship fire detection. Therefore, this embodiment proposes to construct a ship fire detection network LFADF-Net, input the ship fire training data set into the ship fire detection LFADF-Net network; according to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model; collect video data of the target ship; and extract fire features and perform ship fire detection on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain ship fire detection results. This implementation includes a lightweight multi-scale feature extraction network (LMFE) and an adaptive dynamic feature fusion network (ADFF) LFADF-Net to achieve high-performance fire detection. LMFE can reduce the redundant information caused by irregular fire samples in the bounding box and improve feature utilization. The ADFF neck consists of a new fusion architecture and text attention mechanism, which can effectively handle multi-scale problems in complex fire backgrounds, and use the limited features extracted by the LMFE backbone to achieve efficient multi-scale feature fusion with a small number of parameters. In the end, LFADF-Net achieved remarkable results, with mAP@50 reaching 86.3%, a model size of only 4.9M, and a detection speed of 133FPS, which outperformed other common target detection models.

[0059] Continue to refer to Figure 2 Based on the above first embodiment of the method, a second embodiment of the ship fire detection method of the present invention is proposed.

[0060] In this embodiment, before step S10, the method further includes: Collect video data of various ship fires; Processing the ship fire video data to obtain ship fire image data, performing data enhancement on the ship fire image data, and constructing a ship fire annotation data set; The ship fire annotated dataset is annotated to obtain a ship fire training dataset.

[0061] It should be noted that the video data collection of various ship fires includes video data of various flames and smoke in different scenes and angles of the ship. Specifically, "different scenes" refer to various parts and functional areas of the ship, such as the cockpit, engine room, deck, cargo hold, living area, etc.

[0062] Due to their different functions, these areas have different materials, layouts, and equipment configurations. Therefore, when a fire occurs, the flame propagation speed, combustion characteristics, and smoke generation may vary significantly. By collecting and analyzing the flame video data in these different scenarios, we can fully understand the specific performance of the ship when a fire occurs, thereby providing a basis for formulating a more scientific fire detection model. Video data from different angles helps to fully understand the dynamic changes and spatial distribution characteristics of the flame. For example, cameras can be set up at horizontal, vertical, and inclined angles to capture the burning conditions of the flame from different directions and the smoke diffusion path. In addition, the spread of the flame on all levels of the ship can be recorded by installing cameras at different heights. Through multi-angle data collection, the actual situation of the fire scene can be reflected more comprehensively and three-dimensionally, providing rich data support for the construction of fire detection models, and further enhancing the depth and breadth of research on ship fire detection models and methods.

[0063] The various ship fire video data are processed to obtain ship fire image data, and the ship fire image data are data enhanced to construct a ship fire annotation data set. To process the ship fire video data, the video data needs to be preprocessed first, including denoising, stabilization processing, and frame difference techniques to improve the quality and availability of the video image. After the preprocessing is completed, key frames are extracted from the preprocessed video. These key frames can represent the different stages and characteristics of the ship fire to form a preliminary ship fire image data set. The ship fire image data is data enhanced to increase the diversity and robustness of the data and enhance the generalization ability of the model. The data enhancement method mainly includes operations such as rotation, translation, scaling, flipping, color adjustment, and noise addition. Through these operations, ship fire images at various angles and different lighting conditions can be generated, thereby simulating various complex situations that may be encountered in practical applications. Finally, the constructed ship fire annotation data set will contain a large amount of data-enhanced image data, which will be used to train and verify fire detection and recognition models to improve the performance and reliability of the models in practical applications.

[0064] The ship fire labeling dataset is labeled using the Labelme tool to obtain a ship fire training dataset. The purpose of labeling the ship fire labeling dataset using the Labelme tool is to accurately identify and calibrate each image containing fire and smoke information, and to ensure that the formed ship fire training dataset is of high quality and completeness. This process includes carefully drawing bounding boxes in each image to ensure that each annotation accurately reflects the location and range of the fire and smoke. Through this series of labeling work, we finally obtained a ship fire training dataset that meets the requirements. This dataset not only contains rich ship fire image information, but also has accurate and detailed labeling information, providing high-quality training samples for the subsequent training of ship fire detection models. In addition, in order to ensure the diversity and comprehensiveness of the dataset, various ship types, fire scales and environmental conditions are covered when collecting and labeling data, so that the trained model has better generalization ability and practicality.

[0065] Furthermore, in this embodiment, after step S40, the following steps are further included: According to the ship fire detection result, the alarm level is adjusted, and an alarm is issued according to the alarm level.

[0066] It is understandable that fire detection is performed on different scenes of the ship, and the alarm device is controlled to perform multi-level alarms according to the fire detection results. If the fire detection result is that there is a fire, the lower computer controls the alarm device to execute the multi-level alarm mechanism, and automatically adjusts the alarm level according to the fire situation, and controls the alarm device to promptly notify the management personnel and firefighters of the fire information through sound, light, and SMS. Specifically, a variety of alarm methods are designed, including sound and light alarms, SMS alarms, etc. In the case of low fire risk, we will use a lighter alarm method to avoid unnecessary panic and harassment. In the case of high fire risk, we will use a more urgent and effective alarm method to ensure the safety of the crew and the ship. In this way, comprehensive monitoring and fire detection of the internal and external environment of the ship can be achieved, thereby improving the safety and reliability of the ship. At the same time, the multi-level alarm mechanism can also ensure that emergency measures are taken in time when the fire risk is high, minimizing the loss of fire to the ship and personnel.

[0067] This embodiment involves a ship fire rapid detection system, such as Fig.12As shown in the figure, the working principle of the ship fire warning and fire linkage processing subsystem is as follows: when any data processing unit in the data processing subsystem detects a possible fire at a ship fire monitoring scene point, the system located in the ship fire monitoring and alarm processing center will automatically trigger a fire warning. At this time, the loudspeaker and broadcasting equipment will issue an emergency alarm to alert the crew and passengers near the fire point so that they can quickly evacuate the dangerous area. At the same time, the fire video information of the flame scene point will also be sent to the mobile phone or mobile monitoring device of the person responsible for fire safety on the ship. By viewing these video images in real time, relevant personnel can better understand the situation on the scene and develop a more effective emergency plan.

[0068] The key technical points of this embodiment are: A lightweight real-time fire detection model for ships is proposed to cope with different stages of fire events. Its main contributions are as follows: First, RSFD is constructed to cope with the difficulty of acquiring real ship fire images. The dataset covers 16 cabin scenes, including flames and smoke of different scales, and environmental conditions such as different lighting and occlusion. Second, the LMFE backbone network mainly composed of GBS, GSME and SimAM attention modules is carefully built to reduce parameters and ensure real-time performance without compromising accuracy. Third, an efficient ADFF module is developed for multi-scale fire detection in complex backgrounds. Fourth, an innovative lightweight real-time LFADF-Net detection model is demonstrated, which can effectively identify small-scale fires in various environments under limited computing resources. This model achieves an impressive mAP@50 of 86.3% with a model size of only 4.9M and a frame rate of 133 FPS, outperforming other state-of-the-art models. In addition, the model outperforms existing methods in detecting small-scale targets and can effectively handle complex scenes such as occlusion and low light levels, thereby supporting real-time detection of complex ship fire incidents.

[0069] In addition, an embodiment of the present invention further provides a storage medium, on which a ship fire detection program is stored. When the ship fire detection program is executed by a processor, the steps of the ship fire detection method described above are implemented.

[0070] In addition, refer to Fig.13 The embodiment of the present invention further provides a ship fire detection device, the ship fire detection device comprising: A construction module 10 is used to construct a ship fire detection network LFADF-Net, and input the ship fire training data set into the ship fire detection LFADF-Net network; A training module 20 is used to optimize the parameters of the ship fire detection LFADF-Net network according to the ship fire training data set by using a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model; A collection module 30, used for collecting video data of a target ship; The detection module 40 is used to extract fire features and detect ship fires on the video data of the target ship using the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

[0071] Other embodiments or specific implementations of the ship fire detection device of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0072] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0073] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a disk, an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0075] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A ship fire detection method, characterized in that: The ship fire detection method comprises: Constructing a ship fire detection network LFADF-Net, and inputting a ship fire training data set into the ship fire detection LFADF-Net network; According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model; Collect video data of target ships; Fire feature extraction and ship fire detection are performed on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

2. The ship fire detection method according to claim 1, characterized in that: Before the step of constructing a ship fire detection network LFADF-Net and inputting a ship fire training data set into the ship fire detection LFADF-Net network, the step further includes: Collect video data of various ship fires; Processing the ship fire video data to obtain ship fire image data, performing data enhancement on the ship fire image data, and constructing a ship fire annotation data set; The ship fire annotated dataset is annotated to obtain a ship fire training dataset.

3. The ship fire detection method according to claim 1, characterized in that: The method of performing fire feature extraction and ship fire detection on the video data of the target ship by using the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result includes: Using the trained LFADF-Net ship fire detection network model, fire feature extraction and ship fire detection are performed on the video data of the target ship, and the location of the fire and smoke and the corresponding confidence threshold are output; Determining whether there is a fire on the target ship according to the location of the fire and smoke and the corresponding confidence threshold; If it is determined that there is a fire, the video data of the target ship is sent to the temporal optical flow model, and the direction and propagation speed of the fire are calculated according to the movement information of the fire and smoke between the frame images in the video data of the target ship; If it is determined that there is no fire, frame sampling detection is performed based on the image input of the differential frame in the video data of the target ship to obtain a fire optical flow estimation result. Based on the fire optical flow estimation result, a fire or smoke optical flow feature vector is obtained to check whether there is a suspicious fire on the target ship.

4. The ship fire detection method according to claim 1, characterized in that: The ship fire detection network LFADF-Net includes: a lightweight feature extraction backbone, an efficient feature fusion module and a yolov8 detection head.

5. The ship fire detection method according to claim 4, characterized in that: The lightweight feature extraction backbone includes a GBS module and a GSME module; the efficient feature fusion module includes a DySample module, a GTME module and a GSDI module; Eliminate redundant information in the ship fire training data set through the GBS module to obtain de-redundant video data; The GSME module is used to compensate the de-redundant video data and extract multi-scale features to obtain a feature map; Adaptively reducing background information in the feature map based on a dynamic range factor by the DySample module; Through the GTME module, the text attention mechanism is used to dynamically guide the image to focus on the irregular area of ​​the fire sample through the text weight; Through the GSDI module, spatial and channel attention mechanisms are applied to capture global channel information and local spatial details respectively, downsample large feature maps to obtain low-level position information, upsample small feature maps to obtain high-level semantic information, and integrate all features; The yolov8 detection head receives the input of the feature map and detects the fire location and confidence.

6. The ship fire detection method according to claim 1, characterized in that: According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained by a pixel-based loss function to obtain a trained LFADF-Net ship fire detection network model, including: According to the ship fire training data set, the parameters in the ship fire detection LFADF-Net network are optimized and trained through the mixed loss function obtained by rectangular box loss, object loss, classification loss and weighted classification loss to obtain a trained LFADF-Net ship fire detection network model.

7. The ship fire detection method according to any one of claims 1 to 6, characterized in that: After extracting fire features and detecting ship fires from the video data of the target ship using the trained LFADF-Net ship fire detection network model to obtain the ship fire detection result, the method further includes: According to the ship fire detection result, the alarm level is adjusted, and an alarm is issued according to the alarm level.

8. A ship fire detection device, characterized in that: The ship fire detection device comprises: A construction module is used to construct a ship fire detection network LFADF-Net, and input the ship fire training data set into the ship fire detection LFADF-Net network; A training module, used for optimizing and training the parameters in the ship fire detection LFADF-Net network according to the ship fire training data set by means of a pixel-based loss function, so as to obtain a trained LFADF-Net ship fire detection network model; A collection module, used to collect video data of the target ship; The detection module is used to extract fire features and detect ship fires on the video data of the target ship through the trained LFADF-Net ship fire detection network model to obtain a ship fire detection result.

9. A ship fire detection device, characterized in that: The ship fire detection device stores a ship fire detection program, and when the ship fire detection program is executed by the processor, the steps of the ship fire detection method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores a ship fire detection program, which, when executed by a processor, implements the steps of the ship fire detection method according to any one of claims 1 to 7.

Images