A method for measuring the visible light shielding area of ​​smoke screen based on probability model and optical flow

Abstract

The present invention provides a method for measuring the visible light obstruction area of ​​a smoke screen based on a probability model and optical flow. This method addresses the problems of existing visible light smoke screen measurement methods, such as complex test equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structures in images. The method combines a probability model with an optical flow method to intelligently identify and extract smoke-obstructed areas in a scene. The present invention extracts smoke-obstructed areas based on a computer vision algorithm, is simple to operate, and does not require additional auxiliary measurement equipment such as benchmarks. It utilizes stable and rich image features, is highly robust, and is not easily affected by lighting changes and scenes. It uses an optical flow method to estimate local grayscale structural changes in the image, enabling accurate assessment of the smoke screen's obstruction effect. For large-area smoke screen measurement tasks, the present invention is economical and easy to implement, can meet task requirements, and has significant application value.

Description

Technical Field

[0001] The present invention relates to the technical field of testing and evaluating the shielding effect of smoke on visible light bands, and in particular to a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow. Background Art

[0002] Traditional methods for measuring the obscuration performance of visible light smoke screens use a method to calculate the smoke screen contrast. The Chinese invention patent with announcement number CN106840397B uses a benchmark as a background reference and compares the foreground and background areas through differential calculation to obtain the smoke screen contrast. However, the differential method relies on auxiliary measurement equipment such as benchmarks, and has high requirements for lighting and scenes. It is easily affected by changes in lighting and scenes, and cannot meet the needs of field tests for large-area smoke screen measurements. At the same time, the differential method considers changes in the average grayscale of a single point or multiple points, ignoring changes in the local grayscale structure of the image. The structure generated by this light and dark pattern is crucial for target recognition.

[0003] Therefore, in order to meet the needs of field tests for large-area smoke screen measurements, a new visible light smoke screen performance measurement method that does not require additional auxiliary measurement equipment, has good robustness, and produces accurate results is urgently needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems of existing visible light smoke measurement methods, such as complex testing equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structure in the image. A new method for measuring the visible light obscuration area of ​​smoke based on probability model and optical flow is proposed, which does not require additional auxiliary measurement equipment, has good robustness and accurate results.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for measuring the visible light obstruction area of ​​a smoke screen based on a probability model and optical flow, the technical solution includes the following steps:

[0007] Step S1. Fitting the grayscale probability distribution of the background pixels in the image without smoke obstruction: When there is no smoke obstruction, the pixel grayscale also varies, and a single grayscale mean cannot fully describe its distribution. Therefore, it is assumed that the image pixel grayscale follows a Gaussian distribution. Before the smoke screen is deployed, several smoke-free images are cached and the grayscale probability distribution of the smoke-free background pixels at each pixel position is fitted;

[0008] Step S2. Calculate the image optical flow field: During the smoke screen deployment, the image exhibits a diffuse and drifting motion state. If the smoke screen obscures a local area of ​​the image, the image in that area appears to be in motion; otherwise, the area is the background image and appears to be stationary. To obtain the occlusion effect of the smoke screen on the local grayscale structure of the image, the fast optical flow method is used to calculate the dense optical flow field of the image to obtain optical flow information and obtain the pixel-by-pixel motion vector.

[0009] Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained in step S2, smoke pixels in the image are marked using the speed of motion to obtain pixel category labels. If the speed of motion in a certain part of the image is too high to identify the background, it indicates that the grayscale structure at that location is obscured by smoke.

[0010] Step S4. Image mean compensation: During the smoke screen application process, in order to reduce the interference of the overall grayscale change of the image on the probability model, the image during the smoke screen application is mean compensated using the pixels in the area not blocked by the smoke screen;

[0011] Step S5. Calculate the smoke obstruction metric using the probability model: After mean compensation, use the fitted background pixel grayscale probability distribution to calculate the smoke obstruction metric in the image. The larger the pixel grayscale probability, the more consistent it is with the grayscale distribution of no smoke obstruction, and the more likely it is that the pixel is no smoke obstruction;

[0012] Step S6. Labeling smoke pixels using the smoke obscuration metric: After calculating the smoke obscuration metric based on the probability model, label the smoke pixels in the image using the metric to obtain pixel category labels;

[0013] Step S7. Smoke pixel labeling integration: Smoke pixels labeled using both optical flow and probabilistic models may be affected by the scene, resulting in mislabeling. Therefore, to achieve stable and reliable labeling results, a fused labeled image is calculated based on the pixel category labels obtained using optical flow information in step S3 and the smoke obscuration metric calculated using the background pixel grayscale probability distribution in step S6.

[0014] Step S8. Image post-processing: Post-process and filter the fused labeled image using mathematical morphology to remove tiny isolated islands and holes based on the smoothness of the smoke screen.

[0015] Step S9. Calculating the shielding area: Counting the smoke screen pixels based on the smoke screen mark image obtained in step S8, and calculating the actual smoke screen shielding area based on the number of smoke screen pixels using geometric projection relationships.

[0016] Furthermore, in the above step S1, the step of fitting the grayscale probability distribution of the smoke-free background pixel at each pixel position using the smoke-free screen occlusion image includes the following steps:

[0017] For pixel position i in the image, assuming that the grayscale distribution of the pixel conforms to the Gaussian distribution when there is no smoke screen occlusion Before releasing the smoke, cache K frames of images and use the cached images to fit the probability distribution of each pixel through maximum likelihood estimation. Let is the cached pixel value of the kth frame at pixel position i, then:

[0018]

[0019]

[0020] Furthermore, in the above step S2, the step of using the fast optical flow algorithm to calculate the image optical flow field includes the following:

[0021] Step S21: For any pixel position x in the image, let vector u be its corresponding optical flow vector, and let U be the dense optical flow field of the entire image. Use the dense inverse search algorithm to calculate the dense optical flow field of the image.

[0022] Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner. ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field. The downsampling factor between scales is 2. For each scale s, it mainly includes 5 steps:

[0023] Step S221: Create a computational grid. Since image blocks are the basic unit for local optical flow calculation, the full image is divided into a series of sub-image blocks with a given block size and step size. These blocks form a uniform computational grid, which serves as the anchor points for calculating the sparse optical flow field.

[0024] Step S222: Initialize the optical flow field. For the coarsest scale, the optical flow field is initialized to 0; for other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale. For the image block i at position x, its initialized optical flow is

[0025] Step S223: Optical flow inverse search. The optical flow of each image block is quickly calculated by the inverse search algorithm. When calculating the optical flow of the image block, the optimal optical flow is obtained by gradual iteration based on gradient descent. Let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is:

[0026]

[0027] Using the least squares method, we can get the update formula:

[0028]

[0029] in, H=∑ x S T S. When updating, there are

[0030] u←u-Δu(5);

[0031] Step S224: Optical flow densification. After calculating the optical flow of the image block, preprocessing is performed to remove outliers, and then a dense optical flow field is obtained by weighted averaging. i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init The formula for weighted average is:

[0032]

[0033] Among them, d i (x)=I t+1 (x+u i )-T(x), normalization factor Z=∑ i λ i,x / max(1,‖d i (x)‖2), λ if and only if image patch i overlaps with pixel position x i,x =1.

[0034] Step S225: Optical flow variational optimization. By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field. Based on the grayscale constancy assumption, gradient constancy assumption, and smoothness assumption, the energy objective function can be obtained as:

[0035] E(U)=∫ Ω δΨ(E I )+γΨ(E G )+αΨ(E s )dx (7);

[0036] in, ∈=0.001, grayscale item Gradient term Smoothness term And there is

[0037]

[0038] Furthermore, in the above step S3, the step of marking the smoke pixels in the image according to the calculated image optical flow field and the speed of the movement includes the following steps:

[0039] When marking smoke pixels, they are divided into two categories, represented by 0 and 1, corresponding to non-smoke pixels and pixels that may be smoke pixels, respectively. When marking, a threshold g0 is set. Let the size of the optical flow vector be g, then the marking category l:

[0040]

[0041] Furthermore, in the above step S4, the step of performing mean compensation on the image when the smoke screen is applied using pixels in the area not blocked by the smoke screen includes the following steps:

[0042] According to the layout of the measurement scene, the smoke-free area in the image is determined, and some pixels in the smoke-free area are selected as compensation calibration pixels, usually the corner area in the image. Using the image cached before the smoke screen is applied, the mean value of the compensation calibration pixels is calculated, recorded as m0. After the kth frame of the smoke screen is applied, the mean value of the compensation calibration pixels is calculated, recorded as m k For the i-th pixel of the k-th frame Compensated pixel value:

[0043]

[0044] Furthermore, in the above step S5, after mean compensation, the method of calculating the smoke obscuration metric value of each pixel in the image using the fitted background pixel grayscale probability distribution is as follows:

[0045] For Gaussian distribution N(μ,σ 2 ), the probability density calculation formula is as follows:

[0046]

[0047] Based on the above formula, we can calculate the probability density of a pixel belonging to the background pixel. The lower the probability density, the greater the possibility that the corresponding pixel is a smoke pixel. Therefore, based on the calculation formula of probability density, in order to characterize the smoke obscuration effect and simplify the calculation, we can derive the calculation formula of the smoke obscuration metric value:

[0048]

[0049] The larger the value, the greater the deviation of the pixel grayscale from the background probability model. From the perspective of the grayscale probability model, the better the smoke shielding effect. According to the fitted pixel-by-pixel probability distribution and the above smoke shielding metric calculation formula, the mean-compensated pixel grayscale value is substituted into the smoke shielding metric to calculate the smoke shielding effect.

[0050] Furthermore, in the above step S6, after the smoke obstruction measurement value is calculated based on the probability model, the method of using the value to mark the smoke pixels in the image is as follows:

[0051] When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, which correspond to the three categories of non-smoke pixels, possible smoke pixels, and smoke pixels. When marking, three thresholds g0, g1, and x are set. o The thresholds g0 and g1 are the smoke screen masking thresholds, and x0 is the pixel grayscale threshold. Let the pixel grayscale value after mean compensation be x, and the smoke screen masking value be g, then label the category l:

[0052]

[0053] The definition of f(x,x0) depends on the prior knowledge of the grayscale of the smoke. When the smoke is black, it is defined as follows:

[0054]

[0055] When the smoke screen is white, it is defined as follows:

[0056]

[0057] Otherwise, define f(x,t)≡0.

[0058] Furthermore, in the above step S7, the method for obtaining the fused pixel category labels by comprehensively utilizing the pixel category labels obtained by the optical flow information and the pixel category labels obtained by the probability model is as follows:

[0059] For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained using the probability model. The fused pixel category label is obtained by the following formula:

[0060]

[0061] Where 1 represents smoke pixels and 0 represents background pixels.

[0062] Furthermore, in the above step S8, the method of post-processing and filtering the fused marked image using mathematical morphology to filter out tiny isolated islands and holes is as follows:

[0063] Mathematical morphology is a commonly used image processing method, which consists of two basic operations: dilation and erosion, which are defined as follows:

[0064]

[0065] Where f is the image to be filtered, and E is the structuring element of the morphological operation. As the definition shows, the structuring element specifies which pixels in the pixel neighborhood need to participate in the operation when filtering.

[0066] Based on the morphological dilation and erosion operations, the morphological opening and closing operations are defined. The morphological opening operation is defined as:

[0067]

[0068] The morphological closing operation is defined as:

[0069]

[0070] Due to the smooth nature of smoke, it is necessary to filter out tiny islands and holes in the mark during image post-processing. This can be achieved by alternating morphological opening and closing operations. The filtering process is defined as:

[0071]

[0072] The structure element E is defined as a 3×3 matrix:

[0073]

[0074] Furthermore, in the above step S9, the method for calculating the shielding area by counting the smoke pixels based on the smoke mark image after post-processing is as follows:

[0075] Let l i is the category label of the pixel at position i after post-processing (1 is a smoke pixel, 0 is a background pixel). Count all the pixels in the image and get the number of smoke pixels N:

[0076]

[0077] According to the number of smoke screen pixels, the actual smoke screen obstruction area can be calculated using the geometric projection relationship. The calculation method of the actual smoke screen obstruction area S is:

[0078] S = αN (22).

[0079] Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] This invention addresses the problems of existing visible light smoke measurement methods, such as complex test equipment, high requirements for lighting and scenes, susceptibility to interference, and neglect of local grayscale structure in the image. By combining a probabilistic model with the optical flow method, this invention intelligently identifies and extracts smoke-obscured areas in a scene. The invention extracts smoke-obscured areas based on a computer vision algorithm, is simple to operate, and does not require additional auxiliary measurement equipment such as benchmarks. It utilizes stable and rich image features, is highly robust, and is not easily affected by changes in lighting and scenes. It uses the optical flow method to estimate changes in the local grayscale structure of the image, enabling accurate assessment of the smoke-obscured effect. For large-area smoke measurement tasks, this invention is economical and easy to implement, can meet task requirements, and has significant application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 A schematic diagram of a test layout for measuring visible light obstruction performance of a smoke screen using an imaging method in a method for measuring visible light obstruction area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of the optical flow intensity of an image processed in a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention;

[0084] Figure 3 A schematic diagram of a smoke screen obstruction measurement value obtained according to a probability model in a method for measuring visible light obstruction area of ​​smoke screen based on a probability model and optical flow provided by an embodiment of the present invention;

[0085] Figure 4 A schematic diagram of the results of each processing step in a method for measuring the visible light shielding area of ​​a smoke screen based on a probability model and optical flow provided by an embodiment of the present invention.

[0086] The serial numbers in the figure are as follows:

[0087] 1. Visible light camera; 2. Smoke screen; 3. Test background. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0089] like Figures 1 to 4 As shown in the figure, this embodiment provides a method for measuring the visible light shielding area of ​​smoke screen based on probability model and optical flow. The visible light shielding area measurement environment of smoke screen is as follows: Figure 1 As shown, the smoke screen 2 is placed in front of the measurement background 3, and the visible light camera 1 is placed in front of the smoke screen. The specific measurement steps are as follows:

[0090] Step S1. Fitting the grayscale probability distribution of the image background pixels without smoke occlusion: When there is no smoke occlusion, the pixel grayscale also changes, and a single grayscale mean cannot fully describe its distribution. Therefore, it is assumed that the image pixel grayscale conforms to the Gaussian distribution. Before the smoke screen is released, cache K frames of images. In this example, the number of cached images K = 10. For pixel position i in the image, it is assumed that the grayscale distribution of the pixel when there is no smoke occlusion conforms to the Gaussian distribution The fitting formula is as follows:

[0091]

[0092] Step S2. Calculate the image optical flow field: During the smoke screen deployment, the image exhibits a diffuse and drifting motion state. If the smoke screen obscures a local area of ​​the image, the image in that area appears to be in motion; otherwise, the area serves as the background image and appears stationary. To capture the occlusion effect of the smoke screen on the local grayscale structure of the image, a fast optical flow method is used to calculate the dense optical flow field of the image using the previous and next frame images to obtain optical flow information and pixel-by-pixel motion vectors. The specific acquisition steps are as follows:

[0093] Step S21: For any pixel position x in the image, let vector u be its corresponding optical flow vector, and let U be the dense optical flow field of the entire image. Use the dense inverse search algorithm to quickly calculate the image optical flow field.

[0094] Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner. ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field. In this example, θ ss =5,θ sf = 1. The downsampling factor between scales is 2. For each scale s, there are five main steps:

[0095] Step S221: Create a computational grid. Since image blocks are the basic unit for local optical flow calculation, the full image is divided into a series of sub-image blocks, given a block size and step size. These blocks form a uniform computational grid, serving as anchor points for calculating the sparse optical flow field. In this example, the block size is 8 and the step size is 3.

[0096] Step S222: Initialize the optical flow field. For the coarsest scale, the optical flow field is initialized to 0; for other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale. For the image block i at position x, its initialized optical flow is

[0097] Step S223: Optical flow inverse search. The optical flow of each image block is quickly calculated by the inverse search algorithm. When calculating the optical flow of the image block, the optimal optical flow is obtained by gradual iteration based on gradient descent. Let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is:

[0098]

[0099] Using the least squares method, we can get the update formula:

[0100]

[0101] in, H=∑ x S T S. When updating, there are

[0102] u←u-Δu(5);

[0103] Step S224: Optical flow densification. After calculating the optical flow of the image block, preprocessing is performed to remove outliers, and then a dense optical flow field is obtained by weighted averaging. i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init The formula for weighted average is:

[0104]

[0105] Among them, d i (x)=I t+1 (x+u i )-T(x), normalization factor Z=

[0106] ∑ i λ i,x / max(1,‖d i (x)‖2), λ if and only if image patch i overlaps with pixel position x i,x =1.

[0107] Step S225: Optical flow variational optimization. By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field. Based on the grayscale constancy assumption, gradient constancy assumption, and smoothness assumption, the energy objective function can be obtained as:

[0108] E(U)=∫ Ω δΨ(E I )+γΨ(E G )+αΨ(E S )dx (7);

[0109] in, ∈=0.001, grayscale item Gradient term Smoothness term And there is

[0110]

[0111] In this example, the weight values ​​δ=5, γ=10, and α=20.

[0112] Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained from the calculated image optical flow field, smoke pixels in the image are marked using the speed of movement to obtain pixel category labels.

[0113] If the local motion speed of an image is high and the background cannot be identified, it means that the grayscale structure at that location is blocked by smoke; Figure 4 The upper left picture in the four-square grid shows the smoke-occluded area detected using the fast optical flow method.

[0114] The formula for marking smoke pixels in an image is as follows:

[0115] When marking, set the threshold g0. In this example, g0 = 1. Let the size of the optical flow vector be g, then mark the category l:

[0116]

[0117] Among them, 0 and 1 represent non-smoke pixels and pixels that may be smoke, respectively.

[0118] Step S4. Image mean compensation: During the smoke screen application process, in order to reduce the interference of the overall grayscale change of the image on the probability model, the image during the smoke screen application is mean compensated using the pixels in the area not blocked by the smoke screen. Using the image cached before the smoke screen application, the mean of the compensated calibration pixels is calculated, recorded as m0. At the kth frame of the smoke screen application, the mean of the compensated calibration pixels is calculated, recorded as m k For the i-th pixel of the k-th frame Compensated pixel value:

[0119]

[0120] In this example, 100 rows of pixels close to the upper and lower edges of the image are selected as compensation calibration pixels.

[0121] Step S5. Calculate the smoke obstruction metric using the probability model: After mean compensation, the smoke obstruction metric in the image is calculated using the fitted background pixel grayscale probability distribution. The larger the pixel grayscale probability, the more consistent it is with the grayscale distribution of no smoke obstruction, and the more likely it is a pixel without smoke obstruction. The formula for calculating the smoke obstruction metric in the image is:

[0122]

[0123] S6. Labeling smoke pixels using the smoke obscuration metric: After calculating the smoke obscuration metric based on the probability model, the smoke pixels in the image are labeled using the metric to obtain pixel category labels. The specific method is as follows:

[0124] When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, which correspond to the three categories of non-smoke pixels, possible smoke pixels, and smoke pixels. When marking, three thresholds g0, g1, and x are set. o The thresholds g0 and g1 are the smoke mask thresholds, and x0 is the pixel grayscale threshold. In this example, g0 = 30, g1 = 500, and x0 = 100. Let the mean-compensated pixel grayscale value be x and the smoke mask value be g, then label the class l:

[0125]

[0126] In this example, f(x,x0) is defined as follows:

[0127]

[0128] like Figure 4 The upper right picture in the four-square grid is the smoke-obscured area detected by the probability model.

[0129] S7. Smoke Pixel Labeling Integration: Smoke pixels labeled using both optical flow information and the probability model may be affected by the scene, resulting in mislabeling. Therefore, to achieve stable and reliable labeling results, a fused labeled image is calculated based on the pixel category labels obtained using optical flow information in step S3 and the smoke obscuration metric calculated using the background pixel grayscale probability distribution in step S6. The steps are as follows:

[0130] For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained using the probability model. The fused pixel category label is obtained by the following formula:

[0131]

[0132] Where 1 represents smoke pixels and 0 represents background pixels.

[0133] like Figure 4 The lower left image in the four-square grid is the smoke-screened area after integrating the optical flow and probability model.

[0134] S8. Image post-processing: Use mathematical morphology to perform post-processing filtering on the fused marker image to obtain a smoke marker image. The specific method is as follows:

[0135] According to the smooth nature of smoke, during image post-processing, tiny islands and holes in the mark are filtered out through alternating morphological opening and closing operations. The filtering process is defined as:

[0136]

[0137] The structure element E is defined as a 3×3 matrix:

[0138]

[0139] The morphological opening operation is defined as:

[0140]

[0141] The morphological closing operation is defined as:

[0142]

[0143] The mathematical morphology dilation and erosion operations are defined as follows:

[0144]

[0145] like Figure 4 The lower right picture in the four-square grid shows the final smoke-covered area after post-processing.

[0146] S9. Calculate the shielding area: Based on the smoke screen marked image after post-processing in step S8, count the smoke screen pixels and calculate the shielding area.

[0147] Let l i is the category label of the pixel at position i after post-processing (1 is a smoke pixel, 0 is a background pixel). Count all the pixels in the image and get the number of smoke pixels N:

[0148]

[0149] According to the number of smoke screen pixels, the actual smoke screen obstruction area can be calculated using the geometric projection relationship. The calculation method of the actual smoke screen obstruction area S is:

[0150] S = αN (22);

[0151] Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

[0152] This invention addresses the challenges of existing visible light smoke measurement methods, which suffer from complex testing equipment, high lighting and scene requirements, susceptibility to interference, and neglect of local grayscale structure in the image. By combining probabilistic models with optical flow methods, this method intelligently identifies and extracts smoke-obscured areas within a scene. This method extracts smoke-obscured areas based on computer vision algorithms, is simple to operate, and requires no additional auxiliary measurement equipment such as benchmarks. It utilizes stable and rich image features, resulting in high robustness and insensitivity to lighting and scene variations. It employs optical flow methods to estimate changes in the image's local grayscale structure, enabling accurate assessment of smoke obscuration effects. This invention is economical and feasible for large-area smoke measurement tasks, meeting mission requirements.

[0153] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for measuring the visible light shielding area of ​​smoke screen based on probability model and optical flow, characterized in that: The following steps are involved: Step S1. Fitting the grayscale probability distribution of background pixels without smoke screen obstruction: Before the smoke screen is applied, a number of smoke-free images are cached and the grayscale probability distribution of smoke-free background pixels at each pixel position is fitted using the smoke-free images; Step S2. Calculate the image optical flow field: During the smoke screen application process, calculate the dense optical flow field of the image to obtain optical flow information and obtain pixel-by-pixel motion vectors; Step S3. Marking smoke pixels using optical flow information: Based on the optical flow information obtained in step S2, the smoke pixels in the image are marked using the speed of the movement to obtain pixel category labels; Step S4. Image mean compensation: using pixels in the area not blocked by the smoke screen to perform mean compensation on the image when the smoke screen is applied; Step S5. Calculate the smoke obscuration metric using a probability model: After mean compensation, calculate the smoke obscuration metric in the image using the background pixel grayscale probability distribution obtained in step S1; Step S6. Marking smoke pixels using the smoke obscuration metric: Marking smoke pixels in the image based on the smoke obscuration metric to obtain pixel category labels; Step S7. Smoke pixel label integration: Based on the pixel category labels obtained in step S3 using the optical flow information and the pixel category labels obtained in step S6 using the smoke obscuration metric calculated using the background pixel grayscale probability distribution, a fused label image is calculated. Step S8. Image post-processing: using mathematical morphology to perform post-processing filtering on the labeled image fused in step S7 to obtain a smoke labeled image; Step S9. Calculate the shielding area: Based on the smoke mark image obtained in step S8, count the smoke pixels and calculate the shielding area.

2. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized by: In step S1, the step of fitting the grayscale probability distribution of the smoke-free background pixel at each pixel position using the smoke-free screen occlusion image includes the following steps: For pixel position i in the image, assuming that the grayscale distribution of the pixel conforms to the Gaussian distribution when there is no smoke screen occlusion Before releasing the smoke, cache K frames of images and use the cached images to fit the probability distribution of each pixel through maximum likelihood estimation; let is the cached pixel value of the kth frame at pixel position i, then:

3. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized by: In step S2, the step of using a fast optical flow algorithm to calculate the dense optical flow field of the image includes the following steps: Step S21: For any pixel position x in the image, let vector u be its corresponding optical flow vector, and let U be the dense optical flow field of the entire image; use the dense inverse search algorithm to quickly calculate the dense optical flow field of the image; Step S22: The dense inverse search algorithm calculates the optical flow field in a multi-scale manner; from the coarsest scale θ ss At the beginning, the calculation result of each scale is used as the initial value of the optical flow field of the next scale. After scale iteration, the most accurate scale θ is finally obtained. sf Optical flow field; the downsampling factor between scales is 2; for each scale s, the following steps are included: Step S221: Creating a computational grid: Since image blocks are the basic units for local optical flow calculation, given the image block size and step size, the entire image is divided into a series of sub-image blocks; the sub-image blocks form a uniform computational grid, which serves as the anchor points for calculating the sparse optical flow field. Step S222: Initializing the optical flow field: For the coarsest scale, the optical flow field is initialized to 0; For other scales, the initial value of the optical flow field is obtained by scaling the optical flow field calculated at the previous scale; For the image block i at position x, its initialized optical flow is Step S223: Optical flow inverse search: quickly calculate the optical flow of each image block through the inverse search algorithm; When calculating the optical flow of an image block, the optimal optical flow is obtained by gradual iteration based on gradient descent; let T represent the previous frame reference image, I t+1 Represents the current frame image, Δu is the update amount of the optical flow vector during iteration, and its objective function is: Using the least squares method, we can get the update formula: in, H=∑ x S T S; when updating, there is u←u-Δu(5); Step S224: Optical flow densification: After calculating the optical flow of the image block, preprocess it to remove outliers, and then obtain a dense optical flow field by weighted averaging. i,init -u i ||2 Points that exceed the image block size are considered outliers and their optical flow is set to the initial value u i,init ; The formula for weighted average is: Among them, d i (x)=I t+1 (x+u i )-T(x), normalization factor Z=∑ i λ i,x / max(1,‖d i (x)‖2), λ if and only if image patch i overlaps with pixel position x i,x =1; Step S225: Optical flow variational optimization: By using the variational optimization method, the energy function is minimized to achieve the optimization of the optical flow field. Based on the grayscale constancy assumption, the gradient constancy assumption, and the smoothness assumption, the energy objective function can be obtained as follows: in, ∈=0.001, grayscale item Gradient term Smoothness term And there is 4. The method for measuring visible light obstruction area of ​​smoke screens based on a probability model and optical flow according to claim 1, characterized in that: In step S3, the step of marking smoke pixels in the image using optical flow information includes the following steps: When marking smoke pixels, they are divided into two categories, represented by 0 and 1, corresponding to non-smoke pixels and pixels that may be smoke pixels respectively; when marking, set the threshold g0; let the size of the optical flow vector be g, then mark category l:

5. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S4, the step of performing mean compensation on the image when the smoke screen is applied using pixels in the area not blocked by the smoke screen includes the following steps: According to the layout of the measurement scene, the smoke-free area in the image is determined, and some pixels in the smoke-free area are selected as compensation calibration pixels, usually the corner area in the image; the image cached before the smoke screen is cast is used to calculate the mean value of the compensation calibration pixels, which is recorded as m0; after the kth frame of the smoke screen is cast, the mean value of the compensation calibration pixels is calculated, which is recorded as m k ; For the i-th pixel of the k-th frame Compensated pixel value:

6. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S5, after mean compensation, the step of calculating the smoke obscuration metric value of each pixel in the image using the fitted background pixel grayscale probability distribution includes the following steps: For Gaussian distribution N(μ,σ 2 ), the probability density calculation formula is as follows: Based on the above formula, the probability density of a pixel belonging to a background pixel can be calculated; The lower the probability density, the greater the possibility that the corresponding pixel is a smoke pixel. Therefore, according to the calculation formula of probability density, in order to characterize the smoke obscuration effect and simplify the calculation, the calculation formula of the smoke obscuration metric value can be derived: The larger the value, the greater the deviation of the pixel grayscale from the background probability model. From the perspective of the grayscale probability model, the better the smoke shielding effect is. According to the fitted pixel-by-pixel probability distribution and the above smoke shielding metric calculation formula, the mean-compensated pixel grayscale value is substituted into the smoke shielding metric to calculate the smoke shielding effect.

7. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1, characterized in that: In step S6, after calculating the smoke obstruction measurement value based on the probability model, the step of using the value to mark the smoke pixels in the image includes the following steps: When marking smoke pixels, they are divided into three categories, represented by 0, 1, and 2, corresponding to the three categories of non-smoke pixels, possible smoke pixels, and smoke pixels respectively; when marking, three thresholds g0, g1, and x are set. o ; The thresholds g0 and g1 are the smoke shielding measurement thresholds, and x0 is the pixel grayscale threshold. Let the pixel grayscale value after mean compensation be x, and the smoke shielding measurement value be g, then mark the category l: The definition of f(x,x0) depends on the prior knowledge of the grayscale of the smoke screen. When the smoke screen is black, it is defined as follows: When the smoke screen is white, it is defined as follows: Otherwise, define f(x,t)≡0.

8. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1 is characterized in that: In step S7, the step of comprehensively utilizing the pixel category labels obtained by the optical flow information and the pixel category labels obtained by the probability model to obtain fused pixel category labels includes the following steps: For any pixel, let l o is the pixel category label obtained using optical flow information, let l p is the pixel category label obtained using the probability model; the fused pixel category label is obtained by the following formula: Where 1 represents smoke pixels and 0 represents background pixels.

9. The method for measuring visible light shielding area of ​​smoke based on probability model and optical flow according to claim 1, characterized in that: In step S8, the step of performing post-processing filtering on the fused marked image using mathematical morphology to filter out tiny isolated islands and holes includes the following steps: Mathematical morphology is a commonly used image processing method, which consists of two basic operations: dilation and erosion, which are defined as follows: Where f is the image to be filtered, and E is the structuring element of the morphological operation. As can be seen from the definition, the structuring element specifies which pixels in the pixel neighborhood need to participate in the operation when filtering. Based on the morphological expansion and corrosion operations, the morphological opening and closing operations are defined. The morphological opening operation is defined as: The morphological closing operation is defined as: Due to the smooth nature of smoke, it is necessary to filter out tiny islands and holes in the mark during image post-processing. This is achieved by alternating morphological opening and closing operations. The filtering process is defined as: The structure element E is defined as a 3×3 matrix:

10. The method for measuring the visible light shielding area of ​​smoke screen based on probability model and optical flow according to claim 1 is characterized in that: In step S9, the smoke screen pixels are counted based on the smoke screen marked image after post-processing, and the actual smoke screen shielding area is calculated based on the number of smoke screen pixels using a geometric projection relationship. The step of calculating the shielding area includes the following steps: Let l i is the category label of the pixel at position i after post-processing (1 is a smoke pixel, 0 is a background pixel); count all pixels in the image to get the number of smoke pixels N: According to the number of smoke screen pixels, the actual smoke screen shielding area can be calculated using the geometric projection relationship. The calculation method of the actual smoke screen shielding area S is: S = αN (22); Where α is the actual area represented by each pixel calculated based on the geometric projection relationship.

Images