Smoke screen visible light shielding area measurement method based on probability model and light flow

Abstract

The invention provides a smoke screen visible light shielding area measuring method based on a probability model and an optical flow, and aims at solving the problems that an existing visible light smoke screen measuring method is complex in testing equipment, high in requirements for illumination and scenes, prone to being interfered and neglecting a local gray level structure in an image and the like. And performing intelligent identification and extraction on the smoke screen shielding area in the scene. The smoke screen shielding area is extracted based on a computer vision algorithm, the operation is simple, and extra auxiliary measurement equipment such as a marker post is not needed; stable and rich image features are adopted, the robustness is high, and the method is not prone to being influenced by illumination changes and scenes; the local gray level structure change of the image is estimated by adopting the optical flow method, and the capability of accurately evaluating the smoke screen shielding effect is achieved. For a large-area smoke screen measurement task, the method is economical and easy to implement, can meet task requirements, and has a relatively high application value.

Description

Technical Field

[0001] The invention relates to the technical field of testing and evaluating the shielding effect of smoke on visible light bands, and in particular to a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow. Background Art

[0002] The traditional method for measuring the shielding performance of visible light smoke screens uses a method to calculate the smoke screen contrast. The Chinese invention patent with announcement number CN106840397B uses a benchmark as a background reference, and compares the foreground area with the background area through differential calculation to obtain the smoke screen contrast. However, the differential method relies on auxiliary measurement equipment such as benchmarks, and has high requirements for lighting and scenes. It is easily affected by changes in lighting and scenes, and it is difficult to meet the needs of field tests for large-area smoke screen measurements. At the same time, the differential method considers the changes in the average grayscale of a single point or multiple points, ignoring the changes in the local grayscale structure of the image, and the structure generated by this light and dark pattern is crucial for target recognition.

[0003] Therefore, in order to meet the needs of field tests for large-area smoke screen measurements, a new visible light smoke screen performance measurement method that does not require additional auxiliary measurement equipment, has good robustness and accurate results is urgently needed. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of existing visible light smoke measurement methods, such as complex test equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structure in the image. A new method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow is proposed, which does not require additional auxiliary measurement equipment, has good robustness and accurate results.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow, the technical solution includes the following steps:

[0007] Step S1. Fitting the grayscale probability distribution of the image background pixels without smoke screen occlusion: When there is no smoke screen occlusion, the pixel grayscale also changes, and a single grayscale mean cannot fully describe its distribution. Therefore, it is assumed that the image pixel grayscale conforms to the Gaussian distribution. Before the smoke screen is released, cache several smoke-free images and fit the grayscale probability distribution of the smoke-free background pixels at each pixel position;

[0008] Step S2. Calculate the image optical flow field: During the smoke screen release process, the image will diffuse and drift. If the smoke screen covers a certain local area of ​​the image, the image in this area will be in motion; otherwise, the area is the background image and will be static. In order to obtain the occlusion effect of the smoke screen on the local grayscale structure of the image, the fast optical flow method is used to calculate the dense optical flow field of the image to obtain the optical flow information and obtain the pixel-by-pixel motion vector.

[0009] Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained in step S2, the smoke pixels in the image are marked using the speed of movement to obtain pixel category labels. If the local movement speed of an image is large and the background cannot be identified, it means that the grayscale structure at that location is blocked by smoke;

[0010] Step S4. Image mean compensation: During the smoke screen application process, in order to reduce the interference of the overall grayscale change of the image on the probability model, the image during the smoke screen application is mean compensated using the pixels in the area without smoke screen occlusion;

[0011] Step S5. Calculate the smoke shielding metric value using the probability model: After mean compensation, use the fitted background pixel grayscale probability distribution to calculate the smoke shielding metric value in the image. The larger the pixel grayscale probability, the more it conforms to the grayscale distribution without smoke shielding, and the more likely it is a pixel without smoke shielding;

[0012] Step S6. Marking smoke pixels using smoke shielding metrics: After calculating the smoke shielding metrics based on the probability model, use the metrics to mark the smoke pixels in the image to obtain pixel category labels;

[0013] Step S7. Comprehensive smoke pixel labeling: Smoke pixels labeled using optical flow and probability models may be disturbed by the scene and mislabeled. Therefore, in order to obtain stable and reliable labeling results, based on the pixel category label obtained using the optical flow information in step S3 and the pixel category label obtained using the smoke shielding metric value calculated using the grayscale probability distribution of background pixels in step S6, the labeled image after the two are fused is calculated;

[0014] Step S8. Image post-processing: Post-process and filter the fused labeled image using mathematical morphology, and filter out tiny islands and holes according to the smoothness of the smoke screen;

[0015] Step S9. Calculate the shielding area: count the smoke pixels according to the smoke mark image obtained in step S8, and calculate the actual smoke shielding area according to the number of smoke pixels using geometric projection relationship.

[0016] Furthermore, in the above step S1, the step of fitting the grayscale probability distribution of the smoke-free background pixel at each pixel position using the smoke-free screen occlusion image includes the following steps:

[0017] For pixel position i in the image, assuming that the grayscale distribution of the pixel conforms to the Gaussian distribution when there is no smoke screen occlusion Before releasing the smoke, cache K frames of images and use the cached images to fit the probability distribution of each pixel through maximum likelihood estimation. is the cached pixel value of the kth frame at pixel position i, then:

[0018]

[0019]

[0020] Furthermore, in the above step S2, the step of using a fast optical flow algorithm to calculate the optical flow field of the image includes the following steps:

[0021] Step S21: For any pixel position x in the image, let vector u be its corresponding optical flow vector, let U be the dense optical flow field of the whole image, and use the dense inverse search algorithm to calculate the dense optical flow field of the image.

[0022] Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner. ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field. The downsampling factor between scales is 2. For each scale s, there are five main steps:

[0023] Step S221: Create a computational grid. Since the image block is the basic unit for calculating the local optical flow of the image, given the image block size and step size, the whole image is divided into a series of sub-image blocks. These image blocks form a uniform computational grid, which is the anchor point for calculating the sparse optical flow field.

[0024] Step S222: Initialize the optical flow field. For the coarsest scale, the optical flow field is initialized to 0; for other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale. For the image block i at position x, its initial optical flow is

[0025] Step S223: Optical flow inverse search. The optical flow of each image block is quickly calculated through the inverse search algorithm. When calculating the optical flow of the image block, the optimal optical flow is obtained by step-by-step iteration based on gradient descent. Let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is:

[0026]

[0027] Using the least squares method, we can get the update formula:

[0028]

[0029] in, H=∑ x S T S. When updating, there are

[0030] u←u-Δu(5);

[0031] Step S224: Optical flow densification. After calculating the optical flow of the image block, preprocessing is performed to remove outliers, and then a dense optical flow field is obtained by weighted averaging. i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init The formula for weighted average is:

[0032]

[0033] Among them, d i (x) = I t+1 (x+u i )-T(x), normalization factor Z=∑ i λ i,x / max(1,‖d i (x)‖2), if and only if image patch i overlaps with pixel position x i,x =1.

[0034] Step S225: Optical flow variational optimization. By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field. Based on the grayscale constancy assumption, gradient constancy assumption, and smoothness assumption, the energy objective function is:

[0035] E(U)=∫ Ω δΨ(E I )+γΨ(E G )+αΨ(E s )dx (7);

[0036] in, ∈=0.001, grayscale term Gradient term Smooth term And there is

[0037]

[0038] Furthermore, in the above step S3, the step of marking the smoke pixels in the image according to the calculated image optical flow field and the speed of the movement includes the following steps:

[0039] When marking smoke pixels, they are divided into two categories, represented by 0 and 1, corresponding to non-smoke pixels and pixels that may be smoke pixels. When marking, set the threshold g0. Let the size of the optical flow vector be g, then mark category l:

[0040]

[0041] Furthermore, in the above step S4, the step of using pixels in the non-smoke-blocked area to perform mean compensation on the image when the smoke is applied includes the following steps:

[0042] According to the layout of the measurement scene, determine the smoke-free area in the image, and select some pixels in the smoke-free area as compensation calibration pixels, usually the corner area in the image. Use the image cached before the smoke screen is released to calculate the mean value of the compensation calibration pixels, recorded as m0. After the kth frame of the smoke screen is released, calculate the mean value of the compensation calibration pixels, recorded as m k For the i-th pixel of the k-th frame Compensated pixel value:

[0043]

[0044] Furthermore, in the above step S5, after mean compensation, the method of calculating the smoke shielding metric value of each pixel in the image by using the fitted background pixel grayscale probability distribution is as follows:

[0045] For Gaussian distribution N(μ,σ 2 ), the probability density calculation formula is as follows:

[0046]

[0047] Based on the above formula, the probability density of a pixel belonging to a background pixel can be calculated. The lower the probability density, the greater the possibility that the corresponding pixel is a smoke pixel. Therefore, according to the calculation formula of probability density, in order to characterize the smoke shielding effect and simplify the calculation, the calculation formula of the smoke shielding metric value can be derived:

[0048]

[0049] The larger the value, the greater the deviation of the pixel grayscale from the background probability model. From the perspective of the grayscale probability model, the better the smoke shielding effect. Based on the fitted pixel-by-pixel probability distribution and the above smoke shielding metric calculation formula, the mean-compensated pixel grayscale value is substituted into the smoke shielding metric to calculate the smoke shielding effect.

[0050] Furthermore, in the above step S6, after the smoke shielding measurement value is calculated based on the probability model, the method of using the value to mark the smoke pixels in the image is as follows:

[0051] When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, corresponding to non-smoke pixels, possible smoke pixels, and smoke pixels. When marking, three thresholds g0, g1, and x are set. o The thresholds g0 and g1 are the smoke shielding measurement thresholds, and x0 is the pixel grayscale threshold. Let the pixel grayscale value after mean compensation be x, and the smoke shielding measurement value be g, then the label category l:

[0052]

[0053] The definition of f(x,x0) depends on the prior knowledge of the grayscale of the smoke. When the smoke is black, it is defined as follows:

[0054]

[0055] When the smoke screen is white, it is defined as follows:

[0056]

[0057] Otherwise, define f(x,t)≡0.

[0058] Furthermore, in the above step S7, the method of obtaining the fused pixel category label by comprehensively utilizing the pixel category label obtained by the optical flow information and the pixel category label obtained by the probability model is as follows:

[0059] For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained using the probability model. The fused pixel category label is obtained by the following formula:

[0060]

[0061] Where 1 represents smoke pixels and 0 represents background pixels.

[0062] Furthermore, in the above step S8, the method of post-processing and filtering the fused marked image using mathematical morphology to filter out tiny isolated islands and holes is as follows:

[0063] Mathematical morphology is a commonly used image processing method, which consists of two basic operations: dilation and erosion, which are defined as follows:

[0064]

[0065] Where f is the image to be filtered, and E is the structural element of the morphological operation. As can be seen from the definition, the structural element specifies which pixels in the pixel neighborhood need to participate in the operation when filtering.

[0066] Based on the morphological dilation and erosion operations, the morphological opening and closing operations are defined. The morphological opening operation is defined as:

[0067]

[0068] The morphological closing operation is defined as:

[0069]

[0070] According to the smooth nature of smoke, it is necessary to filter out tiny islands and holes in the mark during image post-processing. This can be achieved by alternating morphological opening and closing operations. The filtering process is defined as:

[0071]

[0072] The structure element E is defined as a 3×3 matrix:

[0073]

[0074] Furthermore, in the above step S9, the method of calculating the shielding area by counting smoke pixels based on the smoke mark image after post-processing is as follows:

[0075] Let l i is the category label of the pixel at position i after post-processing is completed (1 is a smoke pixel, 0 is a background pixel). Count all pixels in the image and get the number of smoke pixels N:

[0076]

[0077] According to the number of smoke screen pixels, the actual smoke screen shielding area can be calculated using the geometric projection relationship. The calculation method of the actual smoke screen shielding area S is:

[0078] S = αN (22).

[0079] Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] In view of the problems of existing visible light smoke measurement methods, such as complex test equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structures in images, the present invention combines the probability model with the optical flow method to intelligently identify and extract the smoke-obstructed area in the scene. The present invention extracts the smoke-obstructed area based on a computer vision algorithm, is simple to operate, and does not require additional auxiliary measurement equipment such as benchmarks; it uses stable and rich image features, has high robustness, and is not easily affected by lighting changes and scenes; it uses the optical flow method to estimate the local grayscale structure changes of the image, and has the ability to accurately evaluate the smoke-obstructing effect. For large-area smoke measurement tasks, the present invention is economical and easy to implement, can meet task requirements, and has great application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 A schematic diagram of a test layout for measuring visible light smoke shielding performance using a camera method in a method for measuring visible light shielding area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of the intensity of image optical flow processed in a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention;

[0084] Figure 3 A schematic diagram of a smoke shielding measurement value obtained according to a probability model in a smoke visible light shielding area measurement method based on a probability model and optical flow provided by an embodiment of the present invention;

[0085] Figure 4 A schematic diagram of the results of each processing step in a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention.

[0086] The serial numbers in the figure are as follows:

[0087] 1. Visible light camera; 2. Smoke screen; 3. Test background. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0089] like Figures 1 to 4 As shown in the figure, a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow is provided in this embodiment. The visible light shielding area measurement environment of a smoke screen is as follows: Figure 1 As shown, the position of the smoke screen 2 is in front of the measurement background 3, and the visible light camera 1 is in front of the smoke screen position. The specific measurement steps are as follows:

[0090] Step S1. Fitting the grayscale probability distribution of the image background pixels without smoke screen occlusion: When there is no smoke screen occlusion, the pixel grayscale also changes, and a single grayscale mean cannot fully describe its distribution. Therefore, it is assumed that the image pixel grayscale conforms to the Gaussian distribution. Before the smoke screen is released, cache K frames of images. In this example, the number of cached images K = 10. For pixel position i in the image, it is assumed that the grayscale distribution of the pixel when there is no smoke screen occlusion conforms to the Gaussian distribution. The fitting formula is as follows:

[0091]

[0092] Step S2. Calculate the image optical flow field: During the smoke screen release process, the image will diffuse and drift. If the smoke screen covers a certain local area of ​​the image, the image in this area will be in motion; otherwise, the area is the background image and will be static. In order to obtain the occlusion effect of the smoke screen on the local grayscale structure of the image, the fast optical flow method is used to calculate the dense optical flow field of the image using the previous and next frame images to obtain the optical flow information and the pixel-by-pixel motion vector. The specific acquisition steps are as follows:

[0093] Step S21: For any pixel position x in the image, let vector u be its corresponding optical flow vector, let U be the dense optical flow field of the whole image, and use the dense inverse search algorithm to quickly calculate the image optical flow field.

[0094] Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner. ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field. In this example, θ ss =5,θ sf = 1. The downsampling factor between scales is 2. For each scale s, there are five main steps:

[0095] Step S221: Create a computational grid. Since the image block is the basic unit for calculating the local optical flow of the image, given the image block size and step size, the whole image is divided into a series of sub-image blocks. These image blocks constitute a uniform computational grid, which is the anchor point for calculating the sparse optical flow field. In this example, the image block size is 8 and the step size is 3.

[0096] Step S222: Initialize the optical flow field. For the coarsest scale, the optical flow field is initialized to 0; for other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale. For the image block i at position x, its initial optical flow is

[0097] Step S223: Optical flow inverse search. The optical flow of each image block is quickly calculated by the inverse search algorithm. When calculating the optical flow of the image block, the optimal optical flow is obtained by step-by-step iteration based on gradient descent. Let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is:

[0098]

[0099] Using the least squares method, we can get the update formula:

[0100]

[0101] in, H=∑ x S T S. When updating, there are

[0102] u←u-Δu(5);

[0103] Step S224: Optical flow densification. After calculating the optical flow of the image block, preprocessing is performed to remove outliers, and then a dense optical flow field is obtained by weighted averaging. i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init The formula for weighted average is:

[0104]

[0105] Among them, d i (x) = I t+1 (x+u i )-T(x), normalization factor Z=

[0106] ∑ i λ i,x / max(1,‖d i (x)‖2), if and only if image patch i overlaps with pixel position x i,x =1.

[0107] Step S225: Optical flow variational optimization. By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field. Based on the grayscale constancy assumption, gradient constancy assumption, and smoothness assumption, the energy objective function is:

[0108] E(U)=∫ Ω δΨ(E I )+γΨ(E G )+αΨ(E S )dx (7);

[0109] in, ∈=0.001, grayscale term Gradient term Smooth term And there is

[0110]

[0111] In this example, the weight values ​​are δ=5, γ=10, and α=20.

[0112] Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained by calculating the optical flow field of the image, the smoke pixels in the image are marked using the speed of the movement to obtain pixel category labels.

[0113] If the local motion speed of an image is high and the background cannot be identified, it means that the grayscale structure at that location is blocked by smoke; Figure 4 The upper left picture in the four-square grid shows the smoke-occluded area detected using the fast optical flow method.

[0114] The formula for marking smoke pixels in an image is as follows:

[0115] When marking, set the threshold g0. In this example, g0 = 1. Let the size of the optical flow vector be g, then mark the category l:

[0116]

[0117] Among them, 0 and 1 represent non-smoke pixels and pixels that may be smoke, respectively.

[0118] Step S4. Image mean compensation: During the smoke screen application process, in order to reduce the interference of the overall grayscale change of the image on the probability model, the image during the smoke screen application is mean compensated using the pixels in the area without smoke screen occlusion. Using the image cached before the smoke screen application, calculate the mean of the compensated calibration pixels, denoted as m0. When the smoke screen is applied for the kth frame, calculate the mean of the compensated calibration pixels, denoted as m k For the i-th pixel of the k-th frame Compensated pixel value:

[0119]

[0120] In this example, 100 rows of pixels close to the upper and lower edges of the image are selected as compensation calibration pixels.

[0121] Step S5. Calculate the smoke shielding metric value using the probability model: After mean compensation, use the fitted background pixel grayscale probability distribution to calculate the smoke shielding metric value in the image. The larger the pixel grayscale probability, the more it conforms to the grayscale distribution of no smoke shielding, and the more likely it is a pixel without smoke shielding; the formula for calculating the smoke shielding metric value in the image is:

[0122]

[0123] S6. Marking smoke pixels using smoke shielding metrics: After calculating the smoke shielding metrics based on the probability model, use the metrics to mark the smoke pixels in the image to obtain pixel category labels. The specific method is as follows:

[0124] When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, corresponding to non-smoke pixels, possible smoke pixels, and smoke pixels. When marking, three thresholds g0, g1, and x are set. o . The thresholds g0 and g1 are the smoke shielding measurement thresholds, and x0 is the pixel grayscale threshold. In this example, g0 = 30, g1 = 500, and x0 = 100. Let the pixel grayscale value after mean compensation be x, and the smoke shielding measurement value be g, then the label category l:

[0125]

[0126] In this example, f(x,x0) is defined as follows:

[0127]

[0128] like Figure 4 The upper right picture in the four-square grid is the smoke-obstructed area detected by the probability model.

[0129] S7. Comprehensive smoke pixel labeling: Smoke pixels labeled using optical flow information and probability models may be disturbed by the scene and mislabeled. Therefore, in order to obtain stable and reliable labeling results, based on the pixel category labeling obtained using optical flow information in step S3 and the pixel category labeling obtained using the smoke occlusion metric calculated using the background pixel grayscale probability distribution in step S6, the labeled image after the fusion of the two is calculated. The steps are as follows:

[0130] For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained using the probability model. The fused pixel category label is obtained by the following formula:

[0131]

[0132] Where 1 represents smoke pixels and 0 represents background pixels.

[0133] like Figure 4 The lower left picture in the four-square grid is the smoke-screen occlusion area after the comprehensive optical flow and probability model

[0134] S8. Image post-processing: Use mathematical morphology to perform post-processing filtering on the fused marked image to obtain the smoke marked image. The specific method is as follows:

[0135] According to the smooth nature of smoke, when post-processing the image, the tiny islands and holes in the mark are filtered out through alternating morphological opening and closing operations. The filtering process is defined as:

[0136]

[0137] The structure element E is defined as a 3×3 matrix:

[0138]

[0139] The morphological opening operation is defined as:

[0140]

[0141] The morphological closing operation is defined as:

[0142]

[0143] The mathematical morphology dilation and erosion operations are defined as follows:

[0144]

[0145] like Figure 4 The lower right picture in the four-square grid shows the final smoke-covered area after post-processing.

[0146] S9. Calculate the shielding area: Based on the smoke screen marked image after post-processing in step S8, count the smoke screen pixels and calculate the shielding area.

[0147] Let l i is the category label of the pixel at position i after post-processing is completed (1 is a smoke pixel, 0 is a background pixel). Count all pixels in the image and get the number of smoke pixels N:

[0148]

[0149] According to the number of smoke screen pixels, the actual smoke screen shielding area can be calculated using the geometric projection relationship. The calculation method of the actual smoke screen shielding area S is:

[0150] S = αN (22);

[0151] Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

[0152] In view of the problems of existing visible light smoke measurement methods, such as complex test equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structures in images, the present invention combines the probability model with the optical flow method to intelligently identify and extract the smoke-obstructed area in the scene. The present invention extracts the smoke-obstructed area based on a computer vision algorithm, is simple to operate, and does not require additional auxiliary measurement equipment such as benchmarks; it uses stable and rich image features, has high robustness, and is not easily affected by lighting changes and scenes; it uses the optical flow method to estimate the local grayscale structure changes of the image, and has the ability to accurately evaluate the smoke-obstructed effect. For large-area smoke measurement tasks, the present invention is economical and easy to implement and can meet task requirements.

[0153] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow, characterized in that: The following steps are involved: Step S1. Fitting the grayscale probability distribution of background pixels without smoke screen occlusion: before the smoke screen is released, cache a number of smoke-free images, and use the smoke-free images to fit the grayscale probability distribution of smoke-free background pixels at each pixel position; Step S2. Calculate the image optical flow field: during the smoke screen release process, calculate the dense optical flow field of the image to obtain the optical flow information and obtain the pixel-by-pixel motion vector; Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained in step S2, the smoke pixels in the image are marked using the speed of the movement to obtain pixel category labels; Step S4. Image mean compensation: using pixels in the area without smoke screen occlusion to perform mean compensation on the image when the smoke screen is applied; Step S5. Calculate the smoke shielding metric value using the probability model: After mean compensation, use the background pixel grayscale probability distribution obtained in step S1 to calculate the smoke shielding metric value in the image; Step S6. Marking smoke pixels using the smoke shielding metric value: marking smoke pixels in the image based on the smoke shielding metric value to obtain pixel category labels; Step S7. Smoke pixel label synthesis: Based on the pixel category label obtained by using the optical flow information in step S3 and the pixel category label obtained by using the smoke shielding metric value calculated by the grayscale probability distribution of the background pixels in step S6, a label image after the two are fused is calculated; Step S8. Image post-processing: using mathematical morphology to perform post-processing filtering on the fused marked image in step S7 to obtain a smoke mark image; Step S9. Calculate the shielding area: According to the smoke mark image obtained in step S8, count the smoke pixels and calculate the shielding area.

2. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S1, the step of fitting the grayscale probability distribution of the smoke-free background pixel at each pixel position using the smoke-free screen occlusion image includes the following steps: For pixel position i in the image, assuming that the grayscale distribution of the pixel conforms to the Gaussian distribution when there is no smoke screen occlusion Before releasing the smoke, cache K frames of images and use the cached images to fit the probability distribution of each pixel through maximum likelihood estimation; let is the cached pixel value of the kth frame at pixel position i, then:

3. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S2, the step of using a fast optical flow algorithm to calculate a dense optical flow field of an image includes the following steps: Step S21: for any pixel position x in the image, let vector u be its corresponding optical flow vector, let U be the dense optical flow field of the whole image; use the dense inverse search algorithm to quickly calculate the dense optical flow field of the image; Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner; from the coarsest scale θ ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field; the downsampling factor between scales is 2; for each scale s, the following steps are included: Step S221: Create a computational grid: Since the image block is the basic unit for calculating the local optical flow of the image, given the image block size and step size, the whole image is divided into a series of sub-image blocks; the sub-image blocks constitute a uniform computational grid, which is the anchor point for calculating the sparse optical flow field; Step S222: Initialization of optical flow field: For the coarsest scale, the optical flow field is initialized to 0; For other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale; For image block i at position x, its initial optical flow is Step S223: Optical flow inverse search: quickly calculate the optical flow of each image block through an inverse search algorithm; When calculating the optical flow of an image block, the optimal optical flow is obtained by gradual iteration based on gradient descent; let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is: Using the least squares method, we can get the update formula: in, H=∑ x S T S; when updating, there is u←u-Δu(5); Step S224: Optical flow densification: After calculating the optical flow of the image block, preprocessing is performed to remove outliers, and then a dense optical flow field is obtained by weighted averaging. For the vector offset value || u i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init ; The formula for weighted average is: Among them, d i (x) = I t+1 (x+u i )-T(x), normalization factor Z=∑ i λ i,x / max(1,‖d i (x)‖2), if and only if image patch i overlaps with pixel position x i,x =1; Step S225: Optical flow variational optimization: By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field; based on the grayscale constancy assumption, the gradient constancy assumption, and the smoothness assumption, the energy objective function can be obtained as: in, ∈=0.001, grayscale term Gradient term Smooth term And there is 4. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S3, the step of marking smoke pixels in the image using optical flow information includes the following steps: When marking smoke pixels, they are divided into two categories, represented by 0 and 1, corresponding to non-smoke pixels and pixels that may be smoke pixels respectively; when marking, set the threshold g0; let the size of the optical flow vector be g, then mark category l:

5. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S4, the step of using pixels in the non-smoke-blocked area to perform mean compensation on the image when the smoke is applied includes the following steps: According to the layout of the measurement scene, the smoke-free area in the image is determined, and some pixels in the smoke-free area are selected as compensation calibration pixels, usually the corner area in the image; the image cached before the smoke screen is applied is used to calculate the mean value of the compensation calibration pixels, recorded as m0; the kth frame of the smoke screen is applied, and the mean value of the compensation calibration pixels is calculated, recorded as m k ; For the i-th pixel of the k-th frame Compensated pixel value:

6. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S5, after mean compensation, the step of calculating the smoke shielding metric value of each pixel in the image by using the fitted background pixel grayscale probability distribution includes the following: For Gaussian distribution N(μ,σ 2 ), the probability density calculation formula is as follows: Based on the above formula, the probability density of a pixel belonging to a background pixel can be calculated; The lower the probability density, the greater the possibility that the corresponding pixel is a smoke pixel; therefore, according to the calculation formula of the probability density, in order to characterize the smoke shielding effect and simplify the calculation, the calculation formula of the smoke shielding metric value can be derived: The larger the value, the greater the deviation of the pixel grayscale from the background probability model. From the perspective of the grayscale probability model, the better the smoke shielding effect. According to the fitted pixel-by-pixel probability distribution and the above smoke shielding metric calculation formula, the mean-compensated pixel grayscale value is substituted to calculate the smoke shielding metric that represents the smoke shielding effect.

7. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S6, after calculating the smoke shielding metric value based on the probability model, the step of marking the smoke pixels in the image using the value includes the following steps: When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, corresponding to non-smoke pixels, possible smoke pixels, and smoke pixels. When marking, three thresholds g0, g1, and x are set. o ; The thresholds g0 and g1 are the smoke shielding measurement thresholds, and x0 is the pixel grayscale threshold; let the pixel grayscale value after mean compensation be x, and the smoke shielding measurement value be g, then mark the category l: The definition of f(x,x0) depends on the prior knowledge of the grayscale of the smoke; when the smoke is black, it is defined as follows: When the smoke screen is white, it is defined as follows: Otherwise, define f(x,t)≡0.

8. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S7, the step of comprehensively utilizing the pixel category labels obtained by the optical flow information and the pixel category labels obtained by the probability model to obtain the fused pixel category labels includes the following steps: For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained by using the probability model; the fused pixel category label is obtained by the following formula: Where 1 represents smoke pixels and 0 represents background pixels.

9. The method for measuring the visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S8, the step of post-processing and filtering the fused marked image by using mathematical morphology to filter out tiny isolated islands and holes includes the following steps: Mathematical morphology is a commonly used image processing method, which consists of two basic operations: dilation and erosion, which are defined as follows: Where f is the image to be filtered, and E is the structural element of the morphological operation. As can be seen from the definition, the structural element specifies which pixels in the pixel neighborhood need to participate in the operation during filtering. Based on the morphological expansion and corrosion operations, the morphological opening and closing operations are defined. The morphological opening operation is defined as: The morphological closing operation is defined as: According to the smooth nature of smoke, it is necessary to filter out tiny islands and holes in the mark during image post-processing. This is achieved by alternating morphological opening and closing operations. The filtering process is defined as: The structure element E is defined as a 3×3 matrix:

10. The method for measuring the visible light shielding area of ​​smoke screen based on probability model and optical flow according to claim 1 is characterized in that: In step S9, according to the smoke screen marked image after post-processing, the smoke screen pixels are counted, and according to the number of smoke screen pixels, the actual smoke screen shielding area is calculated using the geometric projection relationship. The step of calculating the shielding area includes the following steps: Let l i is the category label of the pixel at position i after post-processing is completed (1 is a smoke pixel, 0 is a background pixel); count all pixels in the image and get the number of smoke pixels N: According to the number of smoke screen pixels, the actual smoke screen shielding area can be calculated using the geometric projection relationship; the calculation method of the actual smoke screen shielding area S is: S = αN (22); Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

Images