Smoke suppression method in extreme environments based on multi-exposure fusion
By using multi-exposure fusion technology and smoke perception model in the space launch site, the smoke occlusion and image overexposure problems in the initial section of the rocket launch are solved, and imaging quality and measurement accuracy are improved.
Patent Information
- Application Number
- CN202411153720.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-21
AI Technical Summary
During the initial stage of the rocket launch, the photoelectric imaging equipment of the space launch site suffered a decline in image quality due to image overexposure and smoke occlusion, and it was impossible to obtain clear rocket body information, which affected the measurement accuracy and commander judgment.
Using the extreme environmental smoke suppression method based on multi-exposure fusion, the image sequence was collected through the spatial region exposure camera, and the smoke perception model was constructed using brightness, Weber contrast, bright and dark channel contrast and variance feature operators. The segmentation environment was thick smoke and medium smoke. , thin smoke and smoke-free areas, use the Retinex principle and multi-scale pyramid fusion method to perform image fusion and noise suppression, and dynamically adjust the fusion weight to achieve smoke suppression.
It significantly improves the imaging quality under complex light and smoke conditions, improves the adaptive rate of smoke suppression, solves the smoke occlusion and high exposure ratio problems in the initial stage of rocket takeoff, and enhances image clarity and measurement data accuracy.
Smart Images

Figure CN118657673B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image measurement technology, and in particular to a method for suppressing smoke in extreme environments based on multi-exposure fusion. Background Art
[0002] With the vigorous development of aerospace technology, the number of space launch activities has increased year by year, the rocket models have been continuously updated, and the launch conditions are complex and diverse, making it increasingly difficult for the optoelectronic imaging equipment at the space launch site to meet the clear imaging requirements of all-time launches. In the initial stage of rocket launch, the main problems currently faced by the optoelectronic imaging equipment at the launch site are: the image near the engine nozzle is overexposed while the rocket body is underexposed; the rocket engine sprays smoke to submerge the rocket body, and it is impossible to obtain clear rocket body information from the smoke. Specifically, it can be summarized as the problem of imaging overexposure and smoke occlusion. These problems will lead to the decline of image quality and the loss of target information, which not only affects the measurement accuracy of the external ballistic data and takeoff offset data, but also interferes with the commander's judgment and decision-making, and cannot provide a reliable and intuitive image basis for possible security and control instructions. High-dynamic image fusion technology and multi-spectral image fusion technology are reliable ways to solve the above problems. They have been verified in the fields of landscape photography, smart transportation, satellite remote sensing and medical imaging, but there are few application cases in the field of space launch, and the current optoelectronic imaging equipment does not have high-dynamic fusion and smoke suppression functions. Therefore, the study of smoke suppression algorithms based on multi-exposure image fusion in the context of the initial stage of space launch has important practical needs for improving the clarity of aerospace measurement and control optical images and the accuracy of measurement data, and assisting commanders in making decisions. At present, certain results have been achieved in multi-exposure image fusion and smoke suppression algorithms, but for extreme environments such as rocket launches, the single use of multi-exposure fusion or smoke suppression algorithms cannot achieve good results.
[0003] Existing methods are still insufficient in utilizing multi-exposure image sequence information, fail to explore the changing characteristics of smoke under different exposures, and have a low smoke suppression adaptability rate. Summary of the invention
[0004] Based on this, it is necessary to provide a smoke suppression method for extreme environments based on multi-exposure fusion that can improve the smoke suppression adaptability rate in response to the above technical problems.
[0005] A method for suppressing smoke in extreme environments based on multi-exposure fusion, the method comprising:
[0006] Install and debug the spatial sub-area exposure camera in extreme environments, and use the spatial sub-area exposure camera to collect image sequences of the current environment;
[0007] The smoke perception model is constructed using four feature operators: brightness, Weber contrast, bright and dark channel contrast, and variance. The smoke distribution in the image sequence is estimated based on the smoke perception model to obtain the smoke distribution estimation result.
[0008] According to the smoke distribution estimation results, the extreme environment is segmented into four areas; the four areas include thick smoke area, medium smoke area, thin smoke area and no smoke area;
[0009] The image sequences of four areas are layered using the Retinex principle. In the illumination layer, adaptive correction is performed based on the brightness mean of different areas and the fusion weights of brightness for pixel sets in different areas are calculated. In the reflection layer, guided filtering is applied to suppress noise and enhance edges at the same time. According to the multi-scale pyramid fusion method and fusion weights as well as the perceptual weights obtained during guided filtering processing, image fusion and image detail restoration are performed on the layered image sequence to achieve smoke suppression.
[0010] The above-mentioned extreme environment smoke suppression method based on multi-exposure fusion first installs and debugs a spatially divided regional exposure camera in an extreme environment, uses the spatially divided regional exposure camera to collect an image sequence of the current environment, and designs an algorithm in combination with the imaging characteristics of the microarray camera to realize the collection of multiple exposure images of the same scene. In view of the smoke distribution characteristics in the environment, this application uses four feature operators, namely brightness, Weber contrast, bright and dark channel contrast, and variance, to construct a smoke perception model, detects smoke distribution from multiple dimensions such as brightness, contrast, and degree of change, and obtains smoke distribution estimation results. The smoke perception model can be independent of fog-free images and predict smoke distribution by analyzing multi-exposure image sequences, providing key prior information for smoke suppression. The extreme environment is segmented according to the smoke distribution estimation results, and the relationship between exposure and regional brightness changes is deeply analyzed using multi-exposure images, so as to obtain the confidence of smoke in each region to provide important guidance for subsequent regional image enhancement and image fusion. Finally, the fusion weights are designed in combination with regional information to achieve adaptive fusion and image restoration. By dynamically adjusting the weights, the image fusion process is optimized, the quality of the final fused image is significantly improved, and the imaging quality under complex lighting and smoke conditions is effectively improved. It can effectively solve the common image degradation problems such as smoke occlusion and high exposure ratio under extreme conditions such as the initial stage of rocket takeoff, achieve smoke suppression and improve the smoke suppression adaptation rate. The fusion algorithm proposed in this application is also applicable to application scenarios such as fire rescue and fuel analysis, and has a wide range of practical value. At the same time, this application has a small amount of calculation and simple equipment layout, is suitable for extreme outdoor environments, and meets certain real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1A schematic diagram of a flow chart of a smoke suppression method in an extreme environment based on multi-exposure fusion in one embodiment;
[0012] Figure 2 A schematic diagram of a framework of a smoke suppression process in an extreme environment based on multi-exposure fusion in one embodiment;
[0013] Figure 3 A schematic diagram of the attenuation array and attenuation rate distribution of a spatially divided exposure camera in one embodiment;
[0014] Figure 4 A schematic diagram of smoke sensing results in another embodiment;
[0015] Figure 5 This is a diagram showing the effect of smoke suppression treatment in one embodiment. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0017] In one embodiment, Figure 1 and Figure 2 As shown, a method for suppressing smoke in extreme environments based on multi-exposure fusion is provided, comprising the following steps:
[0018] Step 102: Install and debug a spatially divided area exposure camera in an extreme environment, and use the spatially divided area exposure camera to collect an image sequence of the current environment.
[0019] A spatially partitioned exposure (SVE) camera is installed to capture image sequences in extreme environments. An integrated layer of an ordered attenuator array is placed before the camera imaging surface. Each macropixel includes four microattenuators with different light fluxes, covering a single pixel on the CMOS sensor. The transmittance of the attenuators is approximately 1:0.55:0.45:0.0025, and the dynamic range of the camera is extended by 52 dB. By utilizing this attenuation array, the SVE camera can capture images with four different exposure levels in a single shot. The attenuation array and attenuation rate distribution of the spatially partitioned exposure camera are shown in Figure 1. Figure 3 shown.
[0020] Step 104, constructing a smoke perception model using four feature operators, namely, brightness, Weber contrast, bright and dark channel contrast, and variance, and estimating the smoke distribution in the image sequence according to the smoke perception model to obtain a smoke distribution estimation result.
[0021] This application innovatively designs a smoke perception model. This model can predict smoke distribution by analyzing multi-exposure image sequences independently of fog-free images, providing key prior information for smoke suppression. This model lays the foundation for subsequent selective multi-exposure fusion.
[0022] The smoke perception model is used to estimate the smoke distribution of the scene. The model calculates the smoke confidence of each pixel position by extracting features from single images and multi-exposure image sequences. The smoke perception results are shown in Figure 2. Figure 4 shown.
[0023] Firstly, for the single image smoke perception features, the grayscale images captured by the SVE camera are used to calculate the smoke related features through brightness information. The brightness information (BI) is used to preliminarily estimate the smoke factor, which takes into account the difference between the grayscale value of the pixel position at different exposures in the image sequence and the average grayscale value of the image.
[0024] (1)
[0025] in, represents the image number in the sequence, Indicates the pixel position at different exposures in the image The gray value of Representing images The average gray value.
[0026] In addition, improved Weber contrast (WC) is used to further optimize the identification of smoke areas.
[0027] (2)
[0028] in, It's pixels The gradient magnitude at .
[0029] Secondly, for multi-exposure image sequences, this application expands the extraction of smoke perception features. Drawing on the dark channel prior (DCP) and bright channel prior (BCP), a method for calculating contrast features (CF) is proposed, and combined with the atmospheric model of industrial scenes, the transmission rate of local blocks and the approximate constant value of atmospheric light are used to obtain the bright and dark channels and calculate:
[0030] (3)
[0031] in and Represent the dark and light channels respectively.
[0032] In addition, by comparing the fluctuation of pixel variance under different exposure conditions, the present application is able to detect the areas affected by smoke in a multi-exposure image sequence. The variance fluctuation (V) is calculated by the difference between the variance of the pixel under different exposures and the image geometric variance.
[0033] (4)
[0034] in, is the variance of pixels in the image at different exposures. represents the geometric variance of the image, which is the average variance of all pixels in the image at different exposures.
[0035] Finally, the smoke distribution perception model combines brightness information, Weber contrast, contrast characteristics, and variance fluctuations to form a comprehensive smoke perception function. .
[0036] (5)
[0037] The weights of each feature in the model are carefully designed to ensure that the model can effectively identify smoke areas and provide accurate smoke distribution information for image enhancement and fusion.
[0038] Step 106, segmenting the extreme environment according to the smoke distribution estimation result to obtain four areas; the four areas include a thick smoke area, a medium smoke area, a thin smoke area and a no smoke area.
[0039] The grayscale histogram of the smoke distribution estimation result is used to segment the scene area. In order to ensure that the input sequence image remains unchanged, the image is divided into four areas: thick smoke area, medium smoke area, thin smoke area and no smoke area. Different areas are discussed and processed separately.
[0040] Step 108, using the Retinex principle to layer the image sequences of the four regions, adaptively correcting the illumination layer based on the brightness mean of different regions and calculating the fusion weights of the brightness for the pixel sets in different regions, applying guided filtering to suppress noise and enhance edges at the reflection layer, and performing image fusion and image detail restoration on the layered image sequence according to the multi-scale pyramid fusion method and fusion weights and the perceptual weights obtained during guided filtering processing to achieve smoke suppression.
[0041] In response to problems such as image degradation and target information loss in extreme environments, this application proposes a smoke suppression method for extreme environments based on multi-exposure fusion. The method designs an algorithm based on hardware characteristics to achieve high-definition imaging in extreme environments such as the initial stage of rocket launch. This method reveals the intrinsic connection between high-dynamic image fusion and smoke suppression, and significantly improves imaging clarity. In this step, the images in different areas are perceptually enhanced, and the illumination layer and reflection layer are adaptively adjusted. Finally, for multi-exposure fusion, different fusion weights of the illumination layer and the reflection layer are calculated respectively, and the adaptive pyramid fusion technology is used to optimize the fusion performance.
[0042] Firstly, the image of each area is divided into illumination layer and detail layer and processed separately; for the illumination layer, adaptive gamma correction is performed based on the brightness mean of different areas; for the detail layer, guided filtering is used to achieve noise filtering.
[0043] At the same time, the fusion weight is further calculated in combination with the segmentation result obtained in step 106, and the weight is calculated from the illumination layer and the detail layer respectively; for the illumination layer weight, the weight is given to the pixel with a smaller cumulative histogram gradient; for the detail layer weight, the perceptual weight of the guided filter is used to realize the perception of information, which specifically includes:
[0044] The fusion weights are adaptively calculated using the regional information obtained in step 106. For the calculation of the weights of the illumination layer, a new weight , which gives pixels greater weight in the range where the cumulative histogram gradient is smaller. Specifically, the weight formula is as follows:
[0045] (6)
[0046] Indicated in strength The gradient of the cumulative histogram at . Since the gradient here does not refer to the local gradient around the pixel, but the global gradient relative to other pixels in a similar intensity range, it is called the global gradient. Through this design, the adaptive fusion method of the present application can more effectively highlight the high dynamic range area in the image while maintaining the detail information, generating a more visually balanced and natural fused image. This method is particularly suitable for processing multi-exposure images, and can achieve a more optimized image fusion effect under different exposure conditions.
[0047] Then the optimal exposure of different areas is additionally considered, and the weights are calculated as follows:
[0048] (7)
[0049] For the detail layer, the perceptual weight obtained during the guided filtering process is used as the weight.
[0050] The enhanced image sequence and weights are adaptively fused, and the final smoke suppression result is obtained by combining multi-scale pyramid and image restoration, as shown in Figure 2. Figure 5 shown.
[0051] The above-mentioned extreme environment smoke suppression method based on multi-exposure fusion first installs and debugs a spatially divided regional exposure camera in an extreme environment, uses the spatially divided regional exposure camera to collect an image sequence of the current environment, and designs an algorithm in combination with the imaging characteristics of the microarray camera to realize the collection of multiple exposure images of the same scene. In view of the smoke distribution characteristics in the environment, this application uses four feature operators, namely brightness, Weber contrast, bright and dark channel contrast, and variance, to construct a smoke perception model, detects smoke distribution from multiple dimensions such as brightness, contrast, and degree of change, and obtains smoke distribution estimation results. The smoke perception model can be independent of fog-free images and predict smoke distribution by analyzing multi-exposure image sequences, providing key prior information for smoke suppression. The extreme environment is segmented according to the smoke distribution estimation results, and the relationship between exposure and regional brightness changes is deeply analyzed using multi-exposure images, so as to obtain the confidence of smoke in each region to provide important guidance for subsequent regional image enhancement and image fusion. Finally, the fusion weights are designed in combination with regional information to achieve adaptive fusion and image restoration. By dynamically adjusting the weights, the image fusion process is optimized, the quality of the final fused image is significantly improved, and the imaging quality under complex lighting and smoke conditions is effectively improved. It can effectively solve the common image degradation problems such as smoke occlusion and high exposure ratio under extreme conditions such as the initial stage of rocket takeoff, achieve smoke suppression and improve the smoke suppression adaptation rate. The fusion algorithm proposed in this application is also applicable to application scenarios such as fire rescue and fuel analysis, and has a wide range of practical value. At the same time, this application has a small amount of calculation and simple equipment layout, is suitable for extreme outdoor environments, and meets certain real-time requirements.
[0052] In one embodiment, installing and debugging the spatially divided area exposure camera includes:
[0053] A regular array of attenuation sheets is covered before the exposure camera imaging surface. The attenuation sheets are grouped into 4 adjacent 2*2 units of 4 pixels. Images of 4 different exposure levels are obtained simultaneously according to the preset transmittance ratio, and the camera aperture and focal length are adjusted to collect the image sequence of the current environment.
[0054] In one embodiment, the smoke distribution in the image sequence is estimated according to the smoke perception model to obtain the smoke distribution estimation result, including:
[0055] Calculate the difference between the grayscale value of the pixel position under different exposures in the image sequence and the average grayscale value of the image to obtain brightness information;
[0056] Calculate the Weber contrast of smoke regions in the image sequence;
[0057] The image sequence is extracted based on the contrast feature method to obtain the smoke perception features;
[0058] Calculate the difference between the variance of the image sequence at different exposures and the image geometric variance to obtain the variance fluctuation;
[0059] A smoke perception function is constructed based on brightness information, Weber contrast, smoke perception characteristics and variance fluctuation, and the smoke distribution estimation result is output using the smoke perception function.
[0060] In one embodiment, the difference between the grayscale value of the pixel position at different exposures in the image sequence and the average grayscale value of the image is calculated to obtain the brightness information:
[0061] ;
[0062] in, represents the image number in the image sequence, represents the total number of images in the image sequence, Indicates the pixel position at different exposures in the image The gray value of Representing images The average gray value of Represents the threshold value, which is used to filter out darker grayscale areas.
[0063] In one embodiment, the Weber contrast of the smoke region in the image sequence is calculated as:
[0064] ;
[0065] in, represents the image number in the image sequence, represents the total number of images in the image sequence, Indicates the pixel position at different exposures in the image The gray value of is the pixel position The gradient magnitude at .
[0066] In one embodiment, feature extraction is performed on the image sequence according to the contrast feature method, and the smoke perception feature is obtained as follows:
[0067] ;
[0068] in, and represent the dark channel and the bright channel respectively.
[0069] In one embodiment, the difference between the variance of the image sequence at different exposures and the image geometric variance is calculated to obtain the variance fluctuation as:
[0070] ;
[0071] in, is the variance of pixels in the image sequence at different exposures, represents the geometric variance of the image, which is the average variance of all pixels in the image at different exposures.
[0072] In one embodiment, a smoke perception function is constructed based on brightness information, Weber contrast, smoke perception characteristics and variance fluctuation:
[0073] ;
[0074] in, and Represent brightness information, Weber contrast, smoke perception characteristics and variance fluctuation, respectively. and Represents the weight value of each feature, and the sum of their weights is 1.
[0075] In one embodiment, the fusion weights of brightness for pixel sets in different regions are calculated as:
[0076] ;
[0077] in, Indicated in strength The gradient of the cumulative histogram at , Represents the total number of pixels in the image.
[0078] In one embodiment, the fusion weights of pixel sets in different regions are calculated by additionally considering the optimal exposure of different regions:
[0079] ;
[0080] in, represents the brightness component of the image, represents the normalization coefficient, represents the weight adjustment coefficient, Represents the average brightness of different areas.
[0081] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0082] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0083] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A smoke suppression method for extreme environments based on multi-exposure fusion, characterized in that: The method comprises: Install and debug the spatial sub-area exposure camera in extreme environments, and use the spatial sub-area exposure camera to collect image sequences of the current environment; A smoke perception model is constructed using four feature operators, namely, brightness information, Weber contrast, smoke perception characteristics, and variance fluctuation; smoke distribution in the image sequence is estimated according to the smoke perception model to obtain a smoke distribution estimation result; Segmenting the extreme environment according to the smoke distribution estimation result to obtain four areas; the four areas include a thick smoke area, a medium smoke area, a thin smoke area and a no smoke area; The image sequences of the four regions are layered using the Retinex principle. In the illumination layer, adaptive correction is performed based on the average brightness of different regions and the fusion weights of the brightness on the pixel sets in different regions are calculated as follows: in, Indicated in strength The gradient of the cumulative histogram at , Represents the total number of pixels in the image; Guided filtering is applied to the reflection layer to suppress noise and enhance edges at the same time. Image fusion and image detail restoration are performed on the layered image sequence according to the multi-scale pyramid fusion method and fusion weights as well as the perceptual weights obtained during guided filtering to achieve smoke suppression. The smoke distribution in the image sequence is estimated according to the smoke perception model to obtain a smoke distribution estimation result, including: Calculating the difference between the grayscale value of the pixel position at different exposures in the image sequence and the average grayscale value of the image to obtain brightness information; Calculating the Weber contrast of the smoke region in the image sequence; Extracting features from the image sequence according to a contrast feature method to obtain smoke perception features; Calculating the difference between the variance of the image sequence at different exposures and the image geometric variance to obtain variance fluctuation; constructing a smoke perception function according to the brightness information, Weber contrast, smoke perception characteristics and variance fluctuation, and outputting a smoke distribution estimation result using the smoke perception function; The smoke perception function is constructed according to the brightness information, Weber contrast, smoke perception characteristics and variance fluctuation: in, , , and Represent brightness information, Weber contrast, smoke perception characteristics and variance fluctuation, respectively. , , and Represents the weight value of each feature, and the sum of their weights is 1.
2. The extreme environment smoke suppression method according to claim 1, characterized in that: The installation and debugging of the spatial sub-area exposure camera includes: A regular array of attenuation sheets is covered before the exposure camera imaging surface. The attenuation sheets are grouped into 4 adjacent 2*2 units of 4 pixels. Images of 4 different exposure levels are obtained simultaneously according to the preset transmittance ratio, and the camera aperture and focal length are adjusted to collect the image sequence of the current environment.
3. The extreme environment smoke suppression method according to claim 1, characterized in that: Calculate the difference between the grayscale value of the pixel position under different exposures in the image sequence and the average grayscale value of the image to obtain the brightness information: in, k represents the image number in the image sequence, represents the total number of images in the image sequence, Indicates the pixel position at different exposures in the image x The gray value of Representing images k The average gray value of Represents the threshold value, which is used to filter out darker grayscale areas.
4. The extreme environment smoke suppression method according to claim 1, characterized in that: The Weber contrast of the smoke region in the image sequence is calculated as in, k represents the image number in the image sequence, represents the total number of images in the image sequence, Indicates the pixel position at different exposures in the image x The gray value of is the pixel position x The gradient magnitude at .
5. The extreme environment smoke suppression method according to claim 1, characterized in that: The image sequence is subjected to feature extraction according to the contrast feature method, and the smoke perception feature is obtained as follows: in, and represent the dark channel and the bright channel respectively.
6. The extreme environment smoke suppression method according to claim 1, characterized in that: Calculate the difference between the variance of the image sequence under different exposures and the image geometric variance, and get the variance fluctuation as in, is the variance of pixels in the image sequence at different exposures, represents the geometric variance of the image, which is the average variance of all pixels in the image at different exposures. Represents the threshold value, which is used to filter out darker grayscale areas.
7. The extreme environment smoke suppression method according to claim 1, characterized in that: The method further comprises: When the optimal exposure of different regions is additionally considered, the fusion weights of pixel sets in different regions are calculated as follows: in, represents the brightness component of the image, represents the normalization coefficient, represents the weight adjustment coefficient, Represents the average brightness of different areas.
