A method and device for fire alarm monitoring of ship enclosed cabins based on machine vision

Abstract

The present invention provides a method and device for fire alarm monitoring in a closed cabin of a ship based on machine vision. The method includes: obtaining a real-time monitoring image inside the engine room of the ship, and judging whether there is a difference between adjacent frames according to the real-time monitoring image inside the engine room of the ship; comparing the fire characteristics of the image frames that are different from the previous frame to judge whether a fire has occurred; when it is judged that a fire has occurred, obtaining a sequence of fire image frames collected by several adjacent cameras to form a sequence of fire image frames, and predicting the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames; generating an optimal escape route according to the fire trend prediction result by using Dijkstra's algorithm. The present invention realizes intelligent disposal such as fire detection, prediction and route planning in a closed cabin of a ship based on the mining and in-depth analysis of visual perception information.

Description

Technical Field

[0001] The present invention relates to the technical fields of ship intelligent operation and maintenance and image processing. Specifically, it particularly relates to a method for monitoring fire in a closed cabin of a ship based on machine vision. Background Art

[0002] With the development of ship intelligence and unmanned operation and the reduction of on-board personnel, video monitoring systems have been widely applied in the field of ship machinery monitoring. With the continuous enhancement of the demand for monitoring the safe operation status of ships and condition-based maintenance, extensive research and application on the in-depth analysis technology of visual perception information have been carried out.

[0003] Traditional fire detectors are easily affected by factors such as the spatial position of sensors in the engine room, light, water mist, and dust, and cannot respond to fires in a timely manner, failing to meet the application requirements of high efficiency, energy conservation, and reliability of ship operation and maintenance systems. Summary of the Invention

[0004] In view of the deficiencies of the prior art, the present invention provides a method and device for monitoring fire in a closed cabin of a ship based on machine vision. The present invention realizes intelligent disposal such as fire detection, prediction, and escape route planning in the closed cabin of the ship by mining and in-depth analyzing visual perception information.

[0005] The technical means adopted by the present invention are as follows:

[0006] A method for monitoring fire in a closed cabin of a ship based on machine vision, comprising:

[0007] Obtain real-time monitoring images inside the ship's engine room, and determine whether there are differences between adjacent frames according to the real-time monitoring images inside the ship's engine room. The real-time monitoring images inside the ship's engine room are extracted by a plurality of cameras installed at fixed positions inside the ship's engine room;

[0008] Perform a comparison of fire characteristics on the image frames that are different from the previous frame to determine whether a fire has occurred;

[0009] When it is determined that a fire has occurred, obtain a sequence of fire image frames formed by a plurality of adjacent cameras collecting fire image frames, and predict the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames;

[0010] Generate an optimal escape route according to the fire trend prediction result by using Dijkstra's algorithm for escape route planning.

[0011] Further, performing a comparison of fire characteristics on the image frames that are different from the previous frame includes: determining whether there is a flame in the current detection environment according to whether there is a brightness different from the previous image frame in a certain area.

[0012] Further, after determining that there is a fire in the current detection environment, preprocess the current image frame, and the preprocessing includes enhancement, filtering, and normalization processing.

[0013] Further, predict the fire trend according to the change of the image features of the fire area in the fire image frame sequence, including:

[0014] Extract the flame color feature according to the color channel feature of the flame pixel area, and the color channel feature of the flame pixel area is that the red channel value of the flame pixel area is greater than the green channel value, and the green channel value is greater than the blue channel value;

[0015] Adopt the statistical method of the second-order moment of the angle to extract the texture feature of the flame image;

[0016] Determine the flame area range based on the flame color feature and texture feature;

[0017] Calculate the area of the flame area in each frame of the image, and use the polynomial fitting method to predict the flame area of the next frame of the image;

[0018] Use the polynomial fitting method to predict the centroid position of the flame area in the next frame of the image.

[0019] Further, according to the fire trend prediction result, use the Dijkstra algorithm to plan the escape route to generate the optimal escape route, including:

[0020] Set the camera inside the cabin as the node (d j , p j ), where d j is the shortest path length from the first starting camera s to point j, and p j is the previous point of point j in the shortest path from s to j. The basic process of solving the shortest path algorithm from the origin point s to j is as follows:

[0021] 1) Initialization, the starting point is set as: ① d s = 0, p s is empty; ② For all other points: d t = ∞, when finding the shortest path for different points, p t is different; ③ Mark the origin point s, record k = s, and set all other points as unmarked;

[0022] 2) Check the distance from all marked points k to their directly connected unmarked points j, and set

[0023] d j = min{d j , d k + l kj} (2)

[0024] where l kj is the direct connection distance from k to j;

[0025] 3) Select the next point. Among all the unmarked nodes, select the smallest i value in d j ;

[0026] d i = min{d j , all unmarked points j} (3)

[0027] The corresponding point i is a point in the shortest path and is set as marked;

[0028] 4) Find the previous point of point i. Among the marked points, find the point u that is directly connected to point i as the previous point, and set i = u;

[0029] 5) Mark point i. If all points are marked, the algorithm exits. Otherwise, record k = i and go back to 1) to continue.

[0030] The present invention also discloses a fire alarm monitoring device for a ship's enclosed cabin based on machine vision, including:

[0031] An image acquisition module, which is used to obtain real-time monitoring images inside the ship's engine room, and judge whether there are differences between adjacent frames according to the real-time monitoring images inside the ship's engine room. The real-time monitoring images inside the ship's engine room are extracted by a number of cameras installed at fixed positions inside the ship's engine room;

[0032] A discovery module, which is used to compare the fire characteristics of the image frames that are different from the previous frame and judge whether a fire has occurred;

[0033] A prediction module, which is used to, when it is judged that a fire has occurred, obtain a sequence of fire image frames composed of fire image frames collected by a number of adjacent cameras, and predict the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames; generate an optimal escape path according to the fire trend prediction result by using Dijkstra's algorithm.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] The present invention provides an intelligent fire alarm disposal method and device for discovering, predicting and planning routes for fires in a ship's enclosed cabin based on computer vision. Combining the state representation of the flame and using color and texture segmentation recognition algorithms, it realizes the accurate recognition and feature extraction of fires in the enclosed cabin. Deeply integrating the intelligent fire alarm disposal of the ship's enclosed cabin based on computer vision with the ship's intelligent operation and maintenance knowledge base, while monitoring the fire, it gives guidance on the safe passage according to the predicted fire trend. Brief Description of the Drawings

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the attached drawings required for the description of the embodiments or the prior art. Obviously, the attached drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other attached drawings can also be obtained based on these attached drawings.

[0037] Figure 1 Schematic diagram of the working process of a ship enclosed cabin fire monitoring method based on machine vision according to the present invention.

[0038] Figure 2 Schematic diagram of the working process of a ship enclosed cabin fire monitoring device based on machine vision according to the present invention. Detailed implementation manners

[0039] In order to enable those skilled in the art of this technology to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the attached drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0040] As Figure 1 shown, the present invention discloses a ship enclosed cabin fire monitoring method based on machine vision, including:

[0041] S1. Obtain real-time monitoring images inside the ship's engine room, and determine whether there are differences between adjacent frames according to the real-time monitoring images inside the ship's engine room. The real-time monitoring images inside the ship's engine room are extracted by a number of cameras installed at fixed positions inside the ship's engine room.

[0042] Specifically, through the visual image or frequency perception information collection cameras distributed in the access channels of the ship's engine room, combined with the types and characteristics of fires on the ship, a depth analysis model with functions such as fire discovery, combustion analysis, and type recognition is established. With the help of the relevant ship fire types and their disposal knowledge in the ship intelligent operation and maintenance knowledge base, the intelligent disposal information of fires in the ship enclosed cabin can be discovered and pushed in a timely manner.

[0043] S2. Compare the fire characteristics of the image frames that are different from the previous frame to determine whether a fire has occurred. It includes: determining whether there is a fire in the current detection environment according to whether there is a brightness different from the previous image frame in a certain area.

[0044] Specifically, the images or videos captured by the cabin passage cameras contain a large amount of information, which affects the calculation efficiency. Based on flame recognition, it is possible to determine whether there is a flame in the current detection environment according to whether there is a brightness different from the previous image frame in a certain area. Then, in order to eliminate interference and increase the stability of the system, it is necessary to preprocess the image to remove useless information before performing subsequent recognition.

[0045] Preprocessing generally includes enhancement, filtering, and thinning. Reducing the image to be preprocessed to the region of interest (ROI) reduces the total amount of image information data to be processed and speeds up the image processing. Then, image normalization is performed to reduce the influence of ambient light on the input image during the recognition process.

[0046] S3. When it is determined that a fire has occurred, obtain the fire image frames captured by several adjacent cameras to form a fire image frame sequence, and predict the fire trend according to the change of the image features of the flame area in the fire image frame sequence. The steps include:

[0047] S301. Extract the flame color features according to the color channel features of the flame pixel area. The color channel features of the flame pixel area are that the red channel value of the flame pixel area is greater than the green channel value, and the green channel value is greater than the blue channel value.

[0048] Specifically, the flame is an important physical feature, and the obvious visible visual features include the color and texture of the flame. The color of the flame varies within the range from red to yellow. The red channel value of the flame pixel area is greater than the green channel value, and the green channel value is greater than the blue channel value. According to the recognized fire image, the deviation is set to ±5°.

[0049] S302. Adopt the statistical method of the second-order moment of the angle to extract the texture features of the flame image.

[0050] In the extraction of texture features, the statistical method of the second-order moment of the angle is adopted. Among them, the definition of the second-order moment of the angle is:

[0051]

[0052] In the formula, N g represents the gray quantization level of the image, and p(i,j) represents the gray-level co-occurrence matrix. It can be seen that a larger value of the second-order moment of the angle indicates a rough texture, while a smaller value indicates a fine texture.

[0053] S303. Determine the range of the flame area based on the flame color features and texture features.

[0054] S304. Calculate the area of the flame area in each frame of the image, and use the polynomial fitting method to predict the flame area of the next frame of the image extracted by the camera.

[0055] Specifically, after determining the flame area, calculate the proportion of the flame area in the entire image. Use polynomial curve fitting on the obtained flame ROI area ratio to obtain the flame pixel area growth curve.

[0056] The fitting curve obtained through polynomial fitting can predict the flame area in the next frame. If you need to predict the change trend of the flame area in the next few frames, you need to add the actual next frame image to correct the fitting curve parameters, so that the prediction result is relatively accurate.

[0057] S305. Use the polynomial fitting method to predict the centroid position of the flame area in the next frame image.

[0058] For detecting moving targets, the centroid position is an important feature. The change in the centroid of the flame can reflect the change trend to a certain extent. Similarly, the calculated centroid coordinates of the flame area can also be curve-fitted using polynomials.

[0059] When judging the development trend of the fire, in order to make the prediction more accurate, combining the moving direction of the centroid coordinates of the flame area and the area change, the flame area in the next frame can be predicted and recursively calculated in sequence according to time.

[0060] S4. Use Dijkstra's algorithm to plan the escape route based on the fire trend prediction result to generate the optimal escape route.

[0061] Specifically, in this application, according to the location of the ignition point and the camera number, use Dijkstra's algorithm to plan the evacuation route, realize the visualization and cloud sharing of the evacuation process, and improve the evacuation effect. Here, the cameras inside the cabin are set as nodes and have a pair of labels (d j , p j ), where d j is the shortest path length from the first starting camera s to point j, and p j is the previous point of point j in the shortest path from s to j. The basic process of solving the shortest path algorithm from the origin point s to j is as follows:

[0062] 1) Initialization, the starting point is set as: ① d s = 0, p s is empty; ② For all other points: d t = ∞, where d t is the shortest path length from the first starting camera s to point t. When finding the shortest path for different points, p t is different; ③ Mark the origin point s, record k = s, and set all other points as unmarked;

[0063] 2) Check the distances from all the marked points k to their directly connected unmarked points j, and set

[0064] d j = min{d j , d k + l kj} (2)

[0065] where l kj is the direct connection distance from k to j, and d k is the shortest length from the first starting camera s to all the marked points k;

[0066] 3) Select the next point. Among all the unmarked nodes, select the smallest i value in d j

[0067] d i = min{d j , all unmarked points j} (3)

[0068] The corresponding point i is a point in the shortest path and is set as marked;

[0069] 4) Find the previous point of point i. Among the marked points, find the point u that is directly connected to point i as the previous point, and set i = u;

[0070] 5) Mark point i. If all points are marked, the algorithm exits; otherwise, record k = i and go back to 1) to continue.

[0071] As Figure 2 shown, the present invention also discloses a ship enclosed cabin fire monitoring device based on machine vision, including:

[0072] An image acquisition module, which is used to obtain real-time monitoring images inside the ship engine room, and judge whether there are differences between adjacent frames according to the real-time monitoring images inside the ship engine room. The real-time monitoring images inside the ship engine room are extracted by several cameras installed at fixed positions inside the ship engine room;

[0073] A discovery module, which is used to compare the fire characteristics of the image frames that are different from the previous frame and judge whether a fire has occurred;

[0074] A prediction module, which is used to, when it is judged that a fire has occurred, obtain a sequence of fire image frames composed of fire image frames collected by several adjacent cameras, and predict the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames; generate an optimal escape path according to the fire trend prediction result by using Dijkstra's algorithm.

[0075] ​Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fire monitoring in a closed cabin of a ship based on machine vision, characterized in that, it includes: Obtain real-time monitoring images inside the ship's engine room, and judge whether there are differences between adjacent frames according to the real-time monitoring images inside the ship's engine room. The real-time monitoring images inside the ship's engine room are extracted by several cameras installed at fixed positions inside the ship's engine room; Compare the fire characteristics of the image frames that are different from the previous frame to judge whether a fire has occurred; When it is judged that a fire has occurred, obtain a sequence of fire image frames collected by several adjacent cameras to form a sequence of fire image frames, and predict the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames, including calculating the area of the flame area of each frame of image, using the polynomial fitting method to predict the flame area of the next frame of image, and using the polynomial fitting method to predict the centroid position of the flame area of the next frame of image; Generate the optimal escape route according to the fire trend prediction result by using the Dijkstra algorithm for escape route planning.

2. The method for fire monitoring in a closed cabin of a ship based on machine vision according to claim 1, characterized in that, Comparing the fire characteristics of the image frames that are different from the previous frame includes: determining whether there is a flame in the current detection environment according to whether there is a brightness different from the previous image frame in a certain area.

3. The method for fire monitoring in a closed cabin of a ship based on machine vision according to claim 2, characterized in that, After determining that there is a flame in the current detection environment, preprocess the current image frame, and the preprocessing includes enhancement, filtering and normalization processing.

4. The method for fire monitoring in a closed cabin of a ship based on machine vision according to claim 1, characterized in that, Predicting the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames further includes: Extracting the flame color characteristics according to the color channel characteristics of the flame pixel area, and the color channel characteristics of the flame pixel area are that the red channel value of the flame pixel area is greater than the green channel value, and the green channel value is greater than the blue channel value; Adopt the statistical method of angular second moment to extract the texture characteristics of the flame image; Determine the flame area range based on the flame color characteristics and texture characteristics.

5. The method for fire monitoring in a closed cabin of a ship based on machine vision according to claim 1, characterized in that, Generating the optimal escape route according to the fire trend prediction result by using the Dijkstra algorithm for escape route planning includes: Set the camera inside the cabin as a node (d j , p j ), where d j is the shortest path length from the first starting camera s to point j, and p j is the previous point of point j in the shortest path from s to j. The basic process of the algorithm for solving the shortest path from the origin point s to j is as follows: 1) Initialize, the starting point is set as: ① d s = 0, p s is empty; ② For all other points: d t = ∞, when finding the shortest path for different points, p t is different; ③ Mark the origin point s, record k = s, and set all other points as unmarked; 2) Check the distance from all the marked points k to their directly connected unmarked points j, and set d j = min{d j , d k + l kj} (2) where l kj is the direct connection distance from k to j; 3) Select the next point. Among all the unmarked nodes, select the one with the smallest i value among d j ​ d i = min{d j , all unlabeled points j} (3) a point in the shortest path of the corresponding point i, and set it as calibrated; 4) Find the previous point of point i, find the point u that is directly connected to point i from the marked points as the previous point, and set i = u; 5) Mark point i. If all points are marked, the algorithm exits, otherwise record k = i and transfer to 1) to continue.

6. A device for fire monitoring in a closed cabin of a ship based on machine vision, characterized in that, it includes: An image acquisition module, which is used to obtain real-time monitoring images inside the ship's engine room, judge whether there are differences between adjacent frames according to the real-time monitoring images inside the ship's engine room, and the real-time monitoring images inside the ship's engine room are extracted by a number of cameras installed at fixed positions inside the ship's engine room; A discovery module, which is used to compare the fire characteristics of the image frames that are different from the previous frame and judge whether a fire has occurred; A prediction module, which is used to, when it is judged that a fire has occurred, obtain a sequence of fire image frames composed of fire image frames collected by a number of adjacent cameras, and predict the fire trend according to the change of the image characteristics of the flame area in the sequence of fire image frames, including calculating the area of the flame area of each frame of image, using the polynomial fitting method to predict the flame area of the next frame of image, and using the polynomial fitting method to predict the centroid position of the flame area of the next frame of image; Generate an optimal escape route according to the fire trend prediction result by using Dijkstra's algorithm for escape route planning.

Images