A method and system for visual online monitoring of smoke in enclosed spaces of ships
By analyzing the grayscale difference and motion changes of the monitoring images in the cabin, calculating the smoke concentration coefficient and transmittance correction factor, combined with the dark channel defog algorithm, the problem that traditional methods are difficult to adapt to the changes in smoke concentration is solved, and the accuracy of smoke monitoring is improved and precision.
Patent Information
- Application Number
- CN202411281247.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-09-13
AI Technical Summary
Traditional dark channel fog removal algorithms are difficult to adapt to the dynamic changes in smoke concentration generated by flame combustion, affecting the accuracy of smoke monitoring.
By acquiring the monitoring images of each frame in the ship's cabin, analyzing the grayscale differences and motion changes of pixel points, extracting the connection domain, calculating the smoke concentration coefficient and transmittance correction factor, and combining the dark channel defog removal algorithm, the monitoring images after defog removal are obtained. Mark the smoke area.
It improves the accuracy and accuracy of smoke monitoring, can better adapt to changes in smoke concentration, and enhances the identification and monitoring of smoke in confined spaces of ships.
Smart Images

Figure CN118781145B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a method and system for visual online monitoring of smoke in a closed space of a ship. Background Art
[0002] The cabins of ships are usually confined spaces. Once a fire occurs, the fire will spread rapidly, causing serious safety accidents. Before the flame burns, smoke is usually produced and diffuses in the cabin. By monitoring the smoke in the cabin, early warning of possible fires in the cabin can be given. With the development of machine vision, vision-based online smoke monitoring can quickly monitor and warn of smoke in the cabin.
[0003] When the flame starts to burn, the area covered by the smoke is usually small. The defogging algorithm is used to defog the monitoring screen of the cabin. By comparing the defogging monitoring screen with the original monitoring screen, the partial monitoring screen of the smoke-covered area can be obtained, thereby realizing online monitoring of the smoke. The dark channel defogging algorithm can effectively defog smoke images in different scenes because of its simple and efficient implementation process. The defogging effect of the dark channel defogging algorithm is related to the setting of the transmittance in the algorithm. The transmittance reflects the ability of light to penetrate the smoke. However, the smoke generated by the flame will diffuse due to the effect of thermal buoyancy, causing the concentration of the smoke generated by the flame to change over time, thereby affecting the light reflection in the smoke-covered area. The dark channel defogging algorithm usually assumes that the transmittance is relatively constant throughout the image, making it difficult for the transmittance obtained in the traditional dark channel defogging algorithm to adapt to the dynamic changes in the smoke concentration generated by the flame, affecting the defogging effect of the cabin monitoring screen, and thus affecting the final smoke monitoring. Summary of the invention
[0004] In order to solve the above technical problems, a method and system for visual online monitoring of smoke in a closed space of a ship are provided to solve the existing problems.
[0005] The solution to the technical problem of the present application is to provide a method and system for visual online monitoring of smoke in a closed space of a ship, comprising the following steps:
[0006] In a first aspect, an embodiment of the present application provides a method for visual online monitoring of smoke in a closed space of a ship, the method comprising the following steps:
[0007] Acquire each frame of monitoring image in the cabin of the ship to obtain each frame of grayscale image;
[0008] Analyze the grayscale difference of pixels between each grayscale image frame and its adjacent grayscale images, and extract connected domains of pixels to obtain all target connected domains of each grayscale image frame; obtain the matching connected domain set of each target connected domain in each grayscale image frame according to the overlap and matching of pixels between each target connected domain in each grayscale image frame and the target connected domains of the previous grayscale images;
[0009] Analyze the differences in motion changes of all pixels in different target connected domains in the matching connected domain set, as well as the similarities in motion changes of different pixels in each target connected domain, to determine the motion feature coefficient of each target connected domain in each frame of the grayscale image;
[0010] According to the change trend of the grayscale value of the pixel points in the target connected domain in the matching connected domain set, the color fading coefficient of each target connected domain in each frame of the grayscale image is determined; in combination with the motion feature coefficient, the smoke suspicion degree of each target connected domain in each frame of the grayscale image is determined, and the smoke connected domain set of each frame of the grayscale image is obtained; the grayscale value of the pixel points in all the target connected domains in the smoke connected domain set and the motion change of the pixel points are analyzed to determine the smoke concentration coefficient of each frame of the grayscale image;
[0011] Based on the smoke concentration coefficient, the correction factor of the transmittance of each frame of the monitoring image is determined, and combined with the dark channel defogging algorithm, the defogging monitoring image is obtained; based on the defogging monitoring image, the grayscale value of each pixel point in the differential binary image corresponding to each frame of the monitoring image is determined, and each frame of the monitoring image is marked to monitor the smoke in the cabin of the ship.
[0012] Preferably, the step of obtaining all target connected domains of each frame of grayscale image includes:
[0013] A three-frame difference method is used to perform difference calculation on each frame of grayscale image and its adjacent grayscale images to obtain a difference image of each frame of grayscale image;
[0014] Based on all the pixels whose grayscale values in the differential image are not 0, connected domains are extracted for all the pixels at the same position in each frame of grayscale image to obtain all the target connected domains in each frame of grayscale image.
[0015] Preferably, the step of obtaining a matching connected domain set of each target connected domain in each frame of the grayscale image includes:
[0016] The previous multiple grayscale images of each grayscale image frame are used to form a local image set of each grayscale image frame; each target connected domain in each grayscale image frame is used as a central connected domain;
[0017] Count the number of pixels with the same coordinates between the central connected domain of each frame of grayscale image and each target connected domain of each frame of grayscale image in its local image set;
[0018] Calculate the sum of the number of pixel points in the central connected domain of each frame of the grayscale image and the number of pixel points in each target connected domain in each frame of the grayscale image in its local image set, and record it as the first sum value;
[0019] The ratio of the number to the first sum is used as the area overlap between the central connected domain of each frame of the grayscale image and each target connected domain in each frame of the grayscale image in the local image set;
[0020] The target connected domain corresponding to the maximum area overlap in each grayscale image frame in the local image set is recorded as a matching connected domain; each target connected domain in each grayscale image frame and all its matching connected domains form a matching connected domain set of each target connected domain in each grayscale image frame.
[0021] Preferably, the step of determining the motion feature coefficient of each target connected domain in each frame of grayscale image includes:
[0022] The dense optical flow algorithm is used to calculate each frame of grayscale image and its previous frame of grayscale image to obtain the optical flow vector of each pixel in the connected domain of each target in each frame of grayscale image;
[0023] The mean value of the optical flow vectors of all pixels in each target connected domain is used as the mean optical flow vector of each target connected domain in each frame of grayscale image;
[0024] Calculate the discrete degree of the modulus of the optical flow mean vector of all target connected domains in the matching connected domain set as the first motion feature value of each target connected domain in each frame of the grayscale image;
[0025] The average of the similarities between the optical flow vectors of all two arbitrary pixels in each target connected domain is taken as the similarity of the optical flow vectors of each target connected domain in each frame of grayscale image;
[0026] Calculating the average of the optical flow vector similarities of all target connected domains in the matching connected domain set as the second motion feature value of each target connected domain in each frame of the grayscale image;
[0027] The ratio of the first motion feature value to the second motion feature value is used as the motion feature coefficient of each target connected domain in each frame of the grayscale image.
[0028] Preferably, determining the color fade coefficient of each target connected domain in each frame of grayscale image includes:
[0029] Performing cluster analysis on all pixel points of all target connected domains in the matching connected domain set of each target connected domain in each frame of grayscale image to obtain multiple clusters;
[0030] The number of frames of the grayscale image corresponding to the pixel points in each cluster and the grayscale value of the pixel points are combined to form a two-dimensional array of each pixel point; a linear fit is performed on the two-dimensional array of all pixel points in each cluster to obtain the slope of the fitting line;
[0031] The reciprocal of the calculation result of the exponential function with the natural constant as the base and the slope as the exponent is calculated; and the average of the reciprocals of all clusters is used as the color fading coefficient of each target connected domain in each frame of the grayscale image.
[0032] Preferably, determining the smoke suspicion degree of each target connected domain in each frame of grayscale image and obtaining the smoke connected domain set of each frame of grayscale image includes:
[0033] The product of the motion feature coefficient and the color fade coefficient is used as the smoke suspicion degree of each target connected domain in each frame of grayscale image;
[0034] The target connected domains whose smoke suspicion degree is greater than a preset threshold value are used to form a smoke connected domain set of each frame of grayscale image.
[0035] Preferably, the determining of the smoke density coefficient of each frame of grayscale image includes:
[0036] Calculate the mean of the grayscale values of all pixels in each target connected domain in the smoke connected domain set of each frame of grayscale image and the mean of the modulus of the streamer vector, which are recorded as the first mean and the second mean;
[0037] The product of the first mean and the second mean is recorded as the first product; the mean of the first product of all target connected domains in the smoke connected domain set is used as the smoke concentration coefficient of each frame of the grayscale image.
[0038] Preferably, the correction factor of the transmittance of each frame of the monitoring image is calculated by the formula: ,in, For the Correction factor for the transmittance of the frame monitoring image, is to preset a first threshold, For the The smoke density coefficient of the frame grayscale image, To preset the second threshold, is the normalization function.
[0039] Preferably, determining the grayscale value of each pixel in the differential binary image corresponding to each frame of monitoring image to obtain each frame of monitoring image after marking includes:
[0040] No. The differential binary image corresponding to the frame monitoring image Gray value of pixel The calculation method is: ,in, For the Frame grayscale image The gray value of a pixel, For the The grayscale image after frame defogging The grayscale value of the pixel point, The grayscale image after defogging is the first The result of graying the monitoring image after frame defogging. is the preset grayscale threshold;
[0041] Based on the pixel points with a gray value of 1 in the differential binary image, the pixel points at the same position in each frame of the monitoring image are marked to obtain each frame of the monitoring image after marking.
[0042] In the second aspect, an embodiment of the present application also provides a visual online smoke monitoring system for a closed space of a ship, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned methods for visual online smoke monitoring for a closed space of a ship are implemented.
[0043] This application has at least the following beneficial effects:
[0044] The present application obtains all target connected domains of each grayscale image frame; obtains a matching connected domain set of each target connected domain in each grayscale image frame, which has the beneficial effect of considering a more accurate division of regions belonging to the same moving target in different grayscale images; determines the motion feature coefficient of each target connected domain in each grayscale image frame, which has the beneficial effect of analyzing the smoke generated by flame combustion and the motion features of the staff to distinguish them, so as to reflect the possibility that the target connected domain corresponds to the smoke generated by flame combustion; determines the color fading coefficient of each target connected domain in each grayscale image frame, which has the beneficial effect of considering the characteristics of the smoke generated by flame combustion that diffuses to the surroundings and the color fading characteristics when diffusing, so as to better distinguish the target connected domain corresponding to the smoke in the future; determines the color fading coefficient of each target connected domain in each grayscale image frame in combination with the motion feature coefficient. Smoke suspicion degree, obtain the set of smoke connected domains of each frame of grayscale image, which has the beneficial effect of improving the distinction between the corresponding areas of smoke and staff in different frames of grayscale images, thereby improving the accuracy of identifying the corresponding areas of smoke in different frames of grayscale images; determine the correction factor of the transmittance of each frame of monitoring image, combine the dark channel defogging algorithm, obtain the defogged monitoring image, determine the grayscale value of each pixel in the differential binary image corresponding to each frame of monitoring image, obtain each marked monitoring image, and monitor the smoke in the cabin of the ship. Its beneficial effect is that it takes into account the smoke concentration of the image, and corrects the transmittance in the dark channel defogging algorithm, so that the transmittance can better compensate for the light attenuation caused by the change in the smoke concentration caused by the flame combustion, so as to more accurately restore the smoke-free image, thereby improving the accuracy of smoke monitoring in the confined space of the ship. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The following is a further detailed description of a visual online smoke monitoring method for a closed space of a ship of the present application in conjunction with the accompanying drawings.
[0046] Figure 1 A flowchart of a method for visual online monitoring of smoke in a closed space of a ship provided in an embodiment of the present application;
[0047] Figure 2 A flowchart of the steps of a method for obtaining motion feature coefficients of each target connected domain in each frame of grayscale image provided by an embodiment of the present application;
[0048] Figure 3 A flowchart of the steps of a method for obtaining the smoke concentration coefficient of each frame of grayscale image provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the following is a further detailed description of a visual online smoke monitoring method and system for a closed space of a ship proposed in the present application in conjunction with the accompanying drawings and implementation examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
[0051] See also Figure 1 , which shows a flowchart of a method for visual online monitoring of smoke in a closed space of a ship provided by an embodiment of the present application, the method comprising the following steps:
[0052] Step 1: Obtain each frame of monitoring image in the cabin of the ship to obtain each frame of grayscale image.
[0053] Cameras are installed at preset positions in the cabin of the ship, wherein the number and positions of the cameras can be set by the implementer. For one of the cameras, multiple frames of monitoring images are continuously and real-time acquired, and each frame of the acquired monitoring image is grayed out and then denoised to reduce the interference of noise in the image, thereby acquiring each frame of monitoring image in the cabin.
[0054] Preferably, in this embodiment, a Gaussian filtering algorithm is used to perform denoising on each frame of grayscale image after grayscale processing, wherein the Gaussian filtering algorithm is a well-known technology and will not be described in detail here. As other implementation methods, the implementer can adopt other methods of the prior art, such as the median filtering algorithm, etc. This embodiment does not impose any special restrictions on this.
[0055] So far, each frame of grayscale image in the cabin is obtained.
[0056] Step 2: Analyze the grayscale difference of pixels between each grayscale image frame and its adjacent grayscale images, and extract connected domains of pixels to obtain all target connected domains of each grayscale image frame; obtain the matched connected domain set of each target connected domain in each grayscale image frame according to the degree of overlap and matching of pixels between each target connected domain in each grayscale image frame and the target connected domains of the previous multiple grayscale images.
[0057] When a flame occurs somewhere in the cabin and produces smoke, the smoke produced by the flame will spread due to the effect of thermal buoyancy, while the other objects in the cabin are usually stationary, such as lighting, cargo, mechanical equipment, etc. Therefore, the difference between two adjacent frames of images is analyzed to determine the connected domain where the moving target is located, specifically:
[0058] A three-frame difference method is used to perform difference calculation on each frame of grayscale image and its adjacent grayscale images to obtain a difference image of each frame of grayscale image;
[0059] Performing connected domain extraction on all pixel points at corresponding positions in each frame of grayscale image for all pixel points whose grayscale values in the differential image are not 0, to obtain all target connected domains in each frame of grayscale image;
[0060] It should be noted that the three-frame difference method and connected domain extraction are well-known technologies and will not be described in detail here. Frame grayscale image, Frame grayscale image, The frame grayscale image is used as the input of the three-frame difference method to obtain the The difference image of the frame grayscale image; Each target connected domain of the frame grayscale image represents the The area where each moving target appears in the frame grayscale image.
[0061] Furthermore, since the two connected domains corresponding to the same moving target in two adjacent grayscale image frames have a high degree of overlap in the area where they are located in the camera shooting picture, the overlap of the connected domains in the two adjacent grayscale image frames is analyzed to determine the area overlap, specifically:
[0062] The previous multiple grayscale images of each grayscale image frame are used to form a local image set of each grayscale image frame;
[0063] Preferably, in this embodiment, the first 20 grayscale images of each grayscale image frame constitute a local image set of each grayscale image frame. As other implementation methods, implementers can set them according to actual conditions.
[0064] Take each target connected domain in each frame of grayscale image as the central connected domain;
[0065] Count the number of pixels with the same coordinates between the central connected domain of each frame of grayscale image and each target connected domain of each frame of grayscale image in its local image set;
[0066] Calculate the sum of the number of pixel points in the central connected domain of each frame of the grayscale image and the number of pixel points in each target connected domain in each frame of the grayscale image in its local image set, and record it as the first sum value;
[0067] The ratio of the number to the first sum is used as the area overlap between the central connected domain of each frame of the grayscale image and each target connected domain in each frame of the grayscale image in the local image set;
[0068] The target connected domain corresponding to the maximum area overlap in each frame of the grayscale image in the local image set is recorded as the matching connected domain;
[0069] Each target connected domain of each frame of grayscale image and all its matching connected domains form a matching connected domain set of each target connected domain in each frame of grayscale image;
[0070] In this embodiment, the statistics Frame grayscale image The target connected domain is connected to the Frame grayscale image The number of pixels with the same coordinates between the target connected domains is recorded as , No. Frame grayscale image The number of pixels in the target connected domain is recorded as , No. Frame grayscale image The number of pixels in the target connected domain is recorded as ,Will , recorded as Frame grayscale image The target connected domain is Frame grayscale image The area overlap between the target connected domains is assumed to be There are H target connected domains in the frame grayscale image. The target connected domain corresponding to the maximum area overlap in the frame grayscale image is taken as the first Frame grayscale image The matching connected domain of the target connected domain; using the above method, calculate the first Frame, frame, , No. The connected domains of each target in the frame grayscale image are Frame grayscale image The area overlap between the target connected domains is selected Frame, frame, , No. The target connected domain corresponding to the maximum area overlap in the frame grayscale image is taken as the first Frame grayscale image The matching connected domain of the target connected domain.
[0071] It should be noted that the regional overlap reflects the degree of overlap between the target connected domains in two frames of grayscale images. The larger the regional overlap, the greater the possibility that the two moving targets corresponding to the connected domains in the two frames of grayscale images belong to the same moving target. If the maximum regional overlap is 0, there is no matching connected domain in the corresponding target connected domain in the grayscale image. The matching connected domain set reflects the set of connected domains corresponding to the moving targets in all grayscale images in the local image set corresponding to the target connected domain of each frame of grayscale image.
[0072] At this point, the matching connected domain set of each target connected domain in each frame of the grayscale image is obtained.
[0073] Step 3: Analyze the differences in motion changes of all pixels in different target connected domains in the matching connected domain set, as well as the similarities in motion changes of different pixels in each target connected domain, to determine the motion feature coefficients of each target connected domain in each frame of the grayscale image.
[0074] Furthermore, when workers move around in the cabin, moving targets will appear in the grayscale image. However, the movement directions of various parts of the workers' bodies usually do not change significantly when they move, while the smoke generated by the burning flame has the characteristic of spreading in all directions, which makes the movement directions of the smoke in different areas very different. In a short period of time, the workers' bodies will move in the same direction, while the smoke will move in multiple directions.
[0075] Based on the above analysis, the moving direction of the pixels in each target connected domain of each frame of grayscale image is analyzed to determine whether the moving target is the smoke generated by the flame combustion. Specifically:
[0076] The dense optical flow algorithm is used to calculate each frame of grayscale image and its previous frame of grayscale image to obtain the optical flow vector of each pixel in the connected domain of each target in each frame of grayscale image;
[0077] The mean value of the optical flow vectors of all pixels in each target connected domain in each frame of grayscale image is used as the mean optical flow vector of each target connected domain in each frame of grayscale image;
[0078] Preferably, in this embodiment, the Farneback dense optical flow algorithm is used to obtain the optical flow vector of each pixel point in the connected domain of each target in each frame of the grayscale image, wherein the Farneback optical flow algorithm is a well-known technology and will not be described here. As other implementation methods, the implementer can adopt other methods of the prior art, such as the Lucas-Kanade optical flow algorithm, etc., and this embodiment does not impose any special restrictions on this.
[0079] It should be noted that the optical flow vector reflects the motion displacement vector of the object position corresponding to each pixel in each frame of the grayscale image compared to the previous frame of the grayscale image, and the optical flow mean vector reflects the motion direction of the moving target corresponding to the corresponding target connected domain. The optical flow vector of each pixel , No. The optical flow vector of each pixel , then the optical flow mean vector is .
[0080] The discrete degree of the modulus of the optical flow mean vectors of all target connected domains in the matching connected domain set is used as the first motion feature value of each target connected domain in each frame of the grayscale image;
[0081] Preferably, in this embodiment, the standard deviation of the modulus of the optical flow mean vector of all target connected domains in the matching connected domain set is used as the first motion feature value of each target connected domain in each frame of grayscale image. As other implementation methods, the implementer can use other methods of the prior art to measure the degree of discreteness, such as variance, coefficient of variation, etc., and this embodiment does not impose any special restrictions on this.
[0082] It should be noted that the first motion feature value reflects the degree of inconsistency between the overall motion direction of the moving target corresponding to each target connected domain in each frame of grayscale image and the image collected before. The larger the first motion feature value, the greater the degree of inconsistency.
[0083] Furthermore, the similarity of the motion directions of the pixels in each target connected domain in each frame of the grayscale image is analyzed to determine the second motion feature value, which is specifically:
[0084] The average of the similarities between the optical flow vectors of all two arbitrary pixels in each target connected domain in each frame of grayscale image is taken as the similarity of the optical flow vectors of each target connected domain in each frame of grayscale image;
[0085] Preferably, in this embodiment, the mean of the cosine similarities between the optical flow vectors of all arbitrary two pixels in each target connected domain in each frame of grayscale image is used as the optical flow vector similarity of each target connected domain in each frame of grayscale image. The method for calculating the cosine similarity is a well-known technology and will not be repeated here. As other implementation methods, the implementer can adopt other methods of the prior art, such as the Pearson correlation coefficient, etc., and this embodiment does not impose any special restrictions on this.
[0086] Taking the average of the optical flow vector similarities of all target connected domains in the matching connected domain set as the second motion feature value of each target connected domain in each frame of the grayscale image;
[0087] It should be noted that the optical flow vector similarity reflects the consistency of the motion directions of each position in the moving target corresponding to the connected domain of each target in each frame of grayscale image. The greater the similarity of the optical flow vector, the more consistent the motion directions; the larger the obtained second motion eigenvalue, the greater the consistency between the motion directions of the moving targets corresponding to the connected domain of each target in each frame of grayscale image and the motion directions of the images collected before.
[0088] Further, based on the first motion feature value and the second motion feature value, the motion feature coefficient of each target connected domain in each frame of the grayscale image is determined to reflect whether the corresponding moving target has the motion change characteristics of the smoke generated by the flame combustion, specifically:
[0089] Using the ratio of the first motion feature value to the second motion feature value as the motion feature coefficient of each target connected domain in each frame of the grayscale image;
[0090] In this embodiment, when calculating the ratio, in order to avoid the denominator being 0, a value greater than 0 is added to the denominator, and the value in this embodiment is 0.01.
[0091] It should be noted that the larger the first motion feature value is, the greater the difference between the motion directions of the various positions of the motion target corresponding to the target connected domain is; the smaller the second motion feature value is, the less the overall motion direction of the motion target corresponding to the matching connected domain in the matching connected domain set is moving in the same direction; the larger the obtained motion feature coefficient is, the more the motion target has the motion change characteristics of smoke generated by flame combustion.
[0092] Furthermore, the flowchart of the steps of the method for obtaining the motion feature coefficients of each target connected domain in each frame of grayscale image provided by the embodiment of the present application is as follows: Figure 2 shown.
[0093] At this point, the motion feature coefficients of each target connected domain in each frame of the grayscale image are obtained.
[0094] Step 4: determine the color fading coefficient of each target connected domain in each frame of grayscale image according to the change trend of the grayscale value of the pixel points in the target connected domain in the matching connected domain set; determine the smoke suspicion degree of each target connected domain in each frame of grayscale image in combination with the motion feature coefficient, and obtain the smoke connected domain set of each frame of grayscale image; analyze the grayscale values of the pixels in all target connected domains in the smoke connected domain set, as well as the motion changes of the pixels, to determine the smoke concentration coefficient of each frame of grayscale image.
[0095] Furthermore, the smoke generated by the burning flame will gradually become thinner in the air during the diffusion process, causing the color of the smoke to gradually fade over time in the diffusion direction of the smoke, while the color of the worker's body does not have this feature when walking; based on the above characteristics, the grayscale value of the pixel points in the target connected domain is analyzed as the image frame number changes, and the color fading coefficient is determined, which is:
[0096] The AP (Affinity Propagation) clustering algorithm is used to perform cluster analysis on all pixel points of all target connected domains in the matching connected domain set of each target connected domain in each frame of grayscale image to obtain multiple clusters.
[0097] It should be noted that the AP clustering algorithm is a well-known technology and will not be described in detail here. The cosine similarity between the streamer vectors of pixel points is used as the similarity between the pixel points, and the AP clustering algorithm is used to perform clustering analysis; each cluster reflects a cluster composed of pixel points with similar motion directions, and a cluster represents a motion direction.
[0098] The frame number of the grayscale image corresponding to the pixel point in each cluster and the grayscale value of the pixel point are combined to form a two-dimensional array of each pixel point;
[0099] Perform linear fitting on the two-dimensional array of all pixels in each cluster to obtain the slope of the fitting line;
[0100] Preferably, in this embodiment, the least squares method is used for linear fitting, wherein the least squares method is a well-known technology and will not be described in detail here. As other implementation methods, the implementer can adopt other methods of the prior art, such as the maximum likelihood estimation method, etc. This embodiment does not impose any special restrictions on this.
[0101] Calculate the reciprocal of the calculation result of the exponential function with the natural constant as the base and the slope as the exponent; take the average of the reciprocals of all clusters as the color fading coefficient of each target connected domain in each frame of the grayscale image;
[0102] It should be noted that the inverse reflects the degree of fading of the color of the moving target over time when the moving target corresponding to the target connected domain moves in a moving direction corresponding to each cluster. The slope is calculated exponentially to prevent the slope of the fitting line from being non-positive. The color fading coefficient reflects the degree of fading of the color of the moving target corresponding to the target connected domain when the color of the moving target moves in all its moving directions. The larger the color fading coefficient, the more the moving target has the color fading characteristics of the smoke produced by flame burning.
[0103] Further, based on the color fade coefficient and the motion feature coefficient, the smoke suspicion degree is determined to reflect the possibility that the moving target corresponding to the target connected domain is smoke generated by flame combustion, specifically:
[0104] The product of the motion feature coefficient and the color fade coefficient is used as the smoke suspicion degree of each target connected domain in each frame of grayscale image;
[0105] It should be noted that the larger the motion feature coefficient is, the more the moving target corresponding to the target connected domain has the motion change characteristics of the smoke produced by flame combustion; the larger the color fading coefficient is, the more the moving target corresponding to the target connected domain has the color fading characteristics of the smoke produced by flame combustion; the greater the obtained smoke suspicion, the greater the possibility that the moving target corresponding to the target connected domain is the smoke produced by flame combustion in the cabin.
[0106] Furthermore, based on the smoke suspicion, the target connected domains corresponding to all smoke regions suspected to be generated by flame combustion are analyzed, specifically:
[0107] The target connected domains whose smoke suspicion degree is greater than a preset threshold are used to form a smoke connected domain set of each frame of grayscale image;
[0108] Preferably, in this embodiment, the preset threshold value is 0.8. As other implementation modes, the implementer can set it according to the actual situation.
[0109] It should be noted that the smoke connected domain set reflects the set of all target connected domains corresponding to all smoke regions suspected to be generated by flame combustion in the cabin.
[0110] Generally speaking, the higher the smoke concentration, the higher the grayscale value of the pixel in the corresponding grayscale image, because the smoke particles will scatter more light, resulting in a decrease in the brightness of the corresponding area in the grayscale image. In addition, when the smoke concentration is high, its movement will be slower because the interaction between particles increases the viscosity of the air, thereby slowing down the diffusion of the smoke.
[0111] Based on the above analysis, the changes in smoke density and diffusion speed are analyzed based on the gray value and streamer vector of the pixel point, and the smoke density coefficient is determined, which is:
[0112] Calculate the mean of the gray values of all pixels in each target connected domain in the smoke connected domain set of each frame of gray image, and record it as the first mean;
[0113] Calculate the mean value of the modulus of the streamer vectors of all pixel points in each target connected domain in the smoke connected domain set of each frame of grayscale image, and record it as the second mean value;
[0114] The product of the first mean and the second mean is recorded as a first product;
[0115] Taking the average of the first products of all target connected domains in the smoke connected domain set of each frame of grayscale image as the smoke concentration coefficient of each frame of grayscale image;
[0116] It should be noted that the first product reflects the overall concentration of the smoke area suspected to be produced by flame combustion corresponding to the target connected domain. The higher the overall brightness of the target connected domain in the grayscale image, that is, the larger the first mean, the faster the overall movement speed of the moving target corresponding to the target connected domain in the grayscale image, that is, the larger the second mean, and the larger the obtained first product, indicating that the overall concentration of the smoke area suspected to be produced by flame combustion corresponding to the target connected domain is greater; the smoke concentration coefficient reflects the overall concentration of all smoke areas suspected to be produced by flame combustion in the cabin. The larger the smoke concentration coefficient, the greater the overall concentration of the smoke area.
[0117] Furthermore, the step flow chart of the method for obtaining the smoke density coefficient of each frame of grayscale image provided in the embodiment of the present application is as follows: Figure 3 shown.
[0118] At this point, the smoke concentration coefficient of each frame of grayscale image is obtained.
[0119] Step 5, based on the smoke concentration coefficient, determine the correction factor of the transmittance of each frame of the monitoring image, and combine the dark channel defogging algorithm to obtain the defogged monitoring image; based on the defogged monitoring image, determine the grayscale value of each pixel in the differential binary image corresponding to each frame of the monitoring image, obtain each frame of the monitoring image after marking, and monitor the smoke in the cabin of the ship.
[0120] Further, based on the smoke concentration coefficient, a correction factor of the transmittance in the dark channel defogging algorithm is determined to perform defogging on the monitoring image, specifically:
[0121] The calculation formula of the correction factor of the transmittance of each frame of monitoring image is: ,in, For the Correction factor for the transmittance of the frame monitoring image, To preset the first threshold, For the The smoke density coefficient of the frame grayscale image, To preset the second threshold, is the normalization function.
[0122] Preferably, in this embodiment, the first threshold is preset The value is 0.9, and the second threshold is preset The value is 0.1. As other implementation methods, the implementer can set it according to the actual situation.
[0123] Based on the correction factor of the transmittance of each frame of monitoring image, a dark channel defogging algorithm is used to defog each frame of monitoring image to obtain a defogged monitoring image;
[0124] It should be noted that the dark channel defogging algorithm is a well-known technology and will not be described in detail here. Secondly, the correction factor of the transmittance The larger the value is, the smaller the transmittance of the dark channel defogging algorithm is. Since the transmittance in the dark channel defogging algorithm reflects the ability of light to penetrate smoke, and the increase in smoke concentration leads to increased light attenuation, it is necessary to compensate for this attenuation by reducing the transmittance in order to more accurately obtain a monitoring image without smoke. At the moment of collecting the frame monitoring image, the greater the overall concentration of all smoke areas suspected to be caused by flame combustion in the cabin, that is, The larger the value is, the smaller the transmittance in the dark channel defogging algorithm should be, that is, the larger the correction factor of the transmittance should be. The bigger.
[0125] Further, based on the monitoring image after defogging, the grayscale value change of the pixel points between the grayscale image after defogging and the grayscale image before defogging is analyzed to determine the grayscale value of each pixel point in the differential binary image, specifically:
[0126] Grayscale each frame of the monitoring image after defogging to obtain a grayscale image of each frame after defogging;
[0127] Calculate the difference between the grayscale value of each pixel in each frame of grayscale image and the grayscale value of the pixel at the corresponding position in the grayscale image after defogging, as the grayscale difference of each pixel in the differential binary image corresponding to each frame of monitoring image;
[0128] If the grayscale difference is greater than or equal to the grayscale threshold, the grayscale value of each pixel in the differential binary image is assigned to 1; if the grayscale difference is less than the grayscale threshold, the grayscale value of each pixel in the differential binary image is assigned to 0;
[0129] In this embodiment, the grayscale value of each pixel in the differential binary image corresponding to each frame of monitoring image is: ,in, For the The differential binary image corresponding to the frame monitoring image The gray value of a pixel, For the Frame grayscale image The gray value of a pixel, For the The grayscale image after frame defogging The gray value of a pixel, is a preset grayscale threshold. In this embodiment, the preset grayscale threshold The value is 16. As other implementation methods, the implementer can set it according to the actual situation.
[0130] It should be noted that, in general, after the dehazing process, the brightness change of the non-smoky area in the grayscale image is usually small, while the brightness change of the smoke area in the smoke area is usually large after the dehazing process. When , all corresponding pixel points in the differential binary image can be regarded as all pixel points in the smoke area suspected to be generated by flame combustion in the cabin.
[0131] Based on the pixel points with a gray value of 1 in the differential binary image, the pixel points at the same position in each frame of the monitoring image are marked to obtain each frame of the monitoring image after marking;
[0132] Each frame of the marked monitoring image is displayed on the monitor to conduct real-time monitoring of smoke in the confined space of the ship.
[0133] Based on the same inventive concept as the above method, an embodiment of the present application also provides a visual online smoke monitoring system for a closed space of a ship, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of any one of the above-mentioned methods for visual online smoke monitoring for a closed space of a ship are implemented.
[0134] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0135] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0136] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the present application. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application without departing from the content of the technical solution of the present application, shall fall within the protection scope of the technical solution of the present application.
Claims
1. A method for visual online monitoring of smoke in a closed space of a ship, characterized in that: The method comprises the following steps: Acquire each frame of monitoring image in the cabin of the ship to obtain each frame of grayscale image; Analyze the grayscale difference of pixels between each grayscale image frame and its adjacent grayscale images, and extract connected domains of pixels to obtain all target connected domains of each grayscale image frame; obtain the matching connected domain set of each target connected domain in each grayscale image frame according to the overlap and matching of pixels between each target connected domain in each grayscale image frame and the target connected domains of the previous grayscale images; Analyze the differences in motion changes of all pixels in different target connected domains in the matching connected domain set, as well as the similarities in motion changes of different pixels in each target connected domain, to determine the motion feature coefficient of each target connected domain in each frame of the grayscale image; According to the change trend of the grayscale value of the pixel points in the target connected domain in the matching connected domain set, the color fading coefficient of each target connected domain in each frame of the grayscale image is determined; in combination with the motion feature coefficient, the smoke suspicion degree of each target connected domain in each frame of the grayscale image is determined, and the smoke connected domain set of each frame of the grayscale image is obtained; the grayscale value of the pixel points in all the target connected domains in the smoke connected domain set and the motion change of the pixel points are analyzed to determine the smoke concentration coefficient of each frame of the grayscale image; Based on the smoke concentration coefficient, the correction factor of the transmittance of each frame of the monitoring image is determined, and combined with the dark channel defogging algorithm, the defogging monitoring image is obtained; based on the defogging monitoring image, the grayscale value of each pixel point in the differential binary image corresponding to each frame of the monitoring image is determined, and each frame of the monitoring image is marked to monitor the smoke in the cabin of the ship.
2. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The step of obtaining all target connected domains of each frame of grayscale image includes: A three-frame difference method is used to perform difference calculation on each frame of grayscale image and its adjacent grayscale images to obtain a difference image of each frame of grayscale image; Based on all the pixels whose grayscale values in the differential image are not 0, connected domains are extracted for all the pixels at the same position in each frame of grayscale image to obtain all the target connected domains in each frame of grayscale image.
3. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The step of obtaining a matching connected domain set of each target connected domain in each frame of the grayscale image includes: The previous multiple grayscale images of each grayscale image frame are used to form a local image set of each grayscale image frame; each target connected domain in each grayscale image frame is used as a central connected domain; Count the number of pixels with the same coordinates between the central connected domain of each frame of grayscale image and each target connected domain of each frame of grayscale image in its local image set; Calculate the sum of the number of pixel points in the central connected domain of each frame of the grayscale image and the number of pixel points in each target connected domain in each frame of the grayscale image in its local image set, and record it as the first sum value; The ratio of the number to the first sum is used as the area overlap between the central connected domain of each frame of the grayscale image and each target connected domain in each frame of the grayscale image in the local image set; The target connected domain corresponding to the maximum area overlap in each grayscale image frame in the local image set is recorded as a matching connected domain; each target connected domain in each grayscale image frame and all its matching connected domains form a matching connected domain set of each target connected domain in each grayscale image frame.
4. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The step of determining the motion feature coefficients of each target connected domain in each frame of grayscale image includes: The dense optical flow algorithm is used to calculate each frame of grayscale image and its previous frame of grayscale image to obtain the optical flow vector of each pixel in the connected domain of each target in each frame of grayscale image; The mean value of the optical flow vectors of all pixels in each target connected domain is used as the mean optical flow vector of each target connected domain in each frame of grayscale image; Calculate the discrete degree of the modulus of the optical flow mean vector of all target connected domains in the matching connected domain set as the first motion feature value of each target connected domain in each frame of the grayscale image; The average of the similarities between the optical flow vectors of all two arbitrary pixels in each target connected domain is taken as the similarity of the optical flow vectors of each target connected domain in each frame of grayscale image; Calculating the average of the optical flow vector similarities of all target connected domains in the matching connected domain set as the second motion feature value of each target connected domain in each frame of the grayscale image; The ratio of the first motion feature value to the second motion feature value is used as the motion feature coefficient of each target connected domain in each frame of the grayscale image.
5. The method for visual online monitoring of smoke in a closed space of a ship according to claim 1, characterized in that: Determining the color fade coefficient of each target connected domain in each frame of grayscale image includes: Performing cluster analysis on all pixel points of all target connected domains in the matching connected domain set of each target connected domain in each frame of grayscale image to obtain multiple clusters; The number of frames of the grayscale image corresponding to the pixel points in each cluster and the grayscale value of the pixel points are combined to form a two-dimensional array of each pixel point; a linear fit is performed on the two-dimensional array of all pixel points in each cluster to obtain the slope of the fitting line; The reciprocal of the calculation result of the exponential function with the natural constant as the base and the slope as the exponent is calculated; and the average of the reciprocals of all clusters is used as the color fading coefficient of each target connected domain in each frame of the grayscale image.
6. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The step of determining the smoke suspicion degree of each target connected domain in each frame of grayscale image and obtaining the smoke connected domain set of each frame of grayscale image includes: The product of the motion feature coefficient and the color fade coefficient is used as the smoke suspicion degree of each target connected domain in each frame of grayscale image; The target connected domains whose smoke suspicion degree is greater than a preset threshold value are used to form a smoke connected domain set of each frame of grayscale image.
7. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 4, characterized in that: The step of determining the smoke density coefficient of each frame of grayscale image comprises: Calculate the mean of the grayscale values of all pixels in each target connected domain in the smoke connected domain set of each frame of grayscale image and the mean of the modulus of the streamer vector, which are recorded as the first mean and the second mean; The product of the first mean and the second mean is recorded as the first product; the mean of the first product of all target connected domains in the smoke connected domain set is used as the smoke concentration coefficient of each frame of the grayscale image.
8. The method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The correction factor of the transmittance of each frame of the monitoring image is calculated as follows: ,in, For the Correction factor for the transmittance of the frame monitoring image, is to preset a first threshold, For the The smoke density coefficient of the frame grayscale image, To preset the second threshold, is the normalization function.
9. A method for visual online monitoring of smoke in a closed space of a ship as claimed in claim 1, characterized in that: The step of determining the grayscale value of each pixel in the differential binary image corresponding to each frame of monitoring image to obtain each frame of monitoring image after marking includes: No. The differential binary image corresponding to the frame monitoring image Gray value of pixel The calculation method is: ,in, For the Frame grayscale image The gray value of a pixel, For the The grayscale image after defogging The grayscale value of the pixel point, The grayscale image after defogging is the first The result of graying the monitoring image after frame defogging. is the preset grayscale threshold; Based on the pixel points with a gray value of 1 in the differential binary image, the pixel points at the same position in each frame of the monitoring image are marked to obtain each frame of the monitoring image after marking.
10. A visual online smoke monitoring system for a closed space of a ship, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method for visual online monitoring of smoke in a closed space of a ship as described in any one of claims 1 to 9 are implemented.
Citation Information
Patent Citations
Image fire hazard detection method based on video monitoring platform
CN106408846A
Smog fire detection method and smog fire detection device
CN106997461A
Building construction safety risk early warning method and system
CN116703787A
Park fire-fighting remote monitoring system based on Internet of Things
CN117058625A
Cited By
Safe sailor cabin structure based on Internet of Things
CN120455483A
