Infrared imaging image denoising and detail enhancement method for harsh environment

Abstract

The present application relates to the technical field of infrared image processing, and discloses an infrared imaging image denoising and detail enhancement method for harsh environment, which comprises the following steps: acquiring a short-time infrared image sequence; detecting saturated pixels and generating saturated source position information; estimating and removing an electronic crosstalk component to generate a crosstalk correction image sequence; calculating a time domain variation feature and generating a crosstalk affected area mask; performing area classification based on the time domain feature; estimating a full-image smoke-free background gray scale; estimating a local transmissivity based on the Beer-Lambert law; performing despeckling recovery to generate a joint denoising image; and performing partitioned dynamic range mapping enhancement to generate a final denoising and detail enhancement infrared image. The present application solves the problem of denoising failure caused by the coupling interference of two types of degradation mechanisms, i.e., strong radiation source electronic crosstalk and continuous smoke.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing technology, and more specifically, to a method for denoising and enhancing details in infrared imaging images for use in harsh environments. Background Technology

[0002] In infrared imaging applications in harsh environments such as continuous explosion battlefield situation monitoring and industrial high-temperature furnace area monitoring, infrared focal plane array detectors need to image composite degradation scenes where extremely strong radiation sources and aerosols such as smoke or dust coexist within the field of view. Strong radiation sources cause detector pixel saturation, and through electronic crosstalk, stripe artifacts are generated in the row and column directions adjacent to the saturation region; continuously released smoke causes overall or localized image contrast attenuation.

[0003] Existing infrared image denoising methods typically handle the two types of degradation mechanisms mentioned above independently: for electronic crosstalk stripe noise, a radial exponential decay halo model centered on the centroid of the saturated region is used for fitting and subtraction; for contrast attenuation caused by smoke, a pre-acquired smoke-free background reference frame is used, and the transmittance is solved by substituting the pixel-by-pixel difference between the current degraded image and the smoke-free background reference frame into the Beer-Lambert law, and then descattering restoration and detail enhancement are performed.

[0004] However, when strong radiation source electronic crosstalk and continuous smoke coexist in the field of view and partially overlap spatially, the aforementioned independent processing methods face coupling failure: the pulse-like gray-level abrupt changes caused by electronic crosstalk lead the smoke region classification algorithm based on temporal variation characteristics to misclassify crosstalk-affected pixels as moving targets, resulting in large-area holes in the smoke transmittance estimation; smoke attenuation causes systematic deviations in the far-end reference background values ​​of the row and column on which the crosstalk attenuation parameter fitting depends; and in continuous smoke release scenarios, it is impossible to obtain an effective smoke-free background reference frame, causing the transmittance estimation method to completely fail. These coupled interferences lead to a severe decrease in denoising accuracy and failure to recover details. Summary of the Invention

[0005] This invention provides a method for denoising and enhancing details of infrared imaging images in harsh environments, solving the technical problem in related technologies that infrared imaging systems are simultaneously subjected to electronic crosstalk noise and smoke scattering interference in harsh environments, and that existing methods cannot jointly suppress the two types of noise and suffer severe loss of detail information.

[0006] This invention discloses a method for denoising and enhancing details of infrared imaging images in harsh environments, comprising: acquiring the current frame and the previous P frames of raw infrared images continuously acquired by an infrared focal plane array detector to form a short-time image sequence; Saturated pixels are detected in each frame of a short-time image sequence and connected component labeling is performed to generate saturation source location information and saturated pixel mask for each frame; For each saturated connected component in each frame of the image, unsaturated pixel grayscale profiles are extracted along the row and column directions, respectively. An exponential decay model is fitted to obtain crosstalk attenuation parameters. Electronic crosstalk components are estimated and subtracted from the original image to generate a crosstalk-corrected image sequence. Calculate the temporal change rate and temporal change variance pixel by pixel for the crosstalk correction image sequence; expand the multi-frame saturated pixel mask along the row and column directions and perform a pixel-by-pixel logical OR operation along the time axis to generate a mask for the crosstalk-affected region. Based on the temporal change rate, temporal change variance, and crosstalk-affected area mask, the pixels of the entire image are classified into three categories: moving target area, smoke dynamic area, and stable area. Based on the classification results of the three types of regions and the crosstalk correction image sequence, background gray level estimation is performed on the three types of regions respectively and combined into a background gray level estimation map of the whole image. Substitute the crosstalk-corrected current frame image and the full-image background grayscale estimation map into the Beer-Lambert law to solve for the local transmittance, and generate a constrained transmittance estimation map after range constraint. The constrained transmittance estimation map is used to perform descattering recovery operation on the current frame image after crosstalk correction, generating a joint denoised image.

[0007] Further, the fitted exponential decay model obtains crosstalk attenuation parameters, estimates and subtracts electronic crosstalk components from the original image, including: For a saturated connected component, extract all unsaturated pixel gray values ​​in the same row as the center row of the saturated connected component to form a row-oriented gray-level profile sequence, and extract all unsaturated pixel gray values ​​in the same column as the center column of the saturated connected component to form a column-oriented gray-level profile sequence. In the row-oriented grayscale profile sequence, select the pixel grayscale values ​​that are more than a preset safe distance from the boundaries of all saturated connected components in the frame, and calculate the midpoint as the row-oriented robust reference background value; perform the same operation on the column-oriented grayscale profile sequence to obtain the column-oriented robust reference background value; The difference between the grayscale value of each unsaturated pixel and the robust baseline background value along the row direction is calculated to form a row difference sequence. The row difference sequence is fitted with an exponential decay function using the least squares method. The exponential decay function means that the difference is equal to the initial amplitude of the row crosstalk multiplied by an exponential function with a negative exponent of the product of the row decay constant and the distance of the pixel from the boundary of the saturated region. The initial amplitude of the row crosstalk and the row decay constant are obtained. The same fitting is performed along the column direction to obtain the initial amplitude of the column crosstalk and the column decay constant. Based on the row and column crosstalk attenuation parameters, the row and column crosstalk estimates for each unsaturated pixel are calculated and superimposed pixel by pixel to generate the electronic crosstalk component estimation map for the saturated source. The electronic crosstalk component estimation maps for all saturated sources in the frame are superimposed pixel by pixel to generate the total crosstalk estimation map for the corresponding frame. The total crosstalk estimation map is subtracted pixel by pixel from the original image and negative values ​​are cropped to zero to generate the crosstalk correction image.

[0008] Furthermore, in the calculation of the row-oriented robust reference background value and the column-oriented robust reference background value, the median value of the far-end pixels in the row-oriented grayscale profile sequence that are more than a preset safe distance from all saturated regions is taken along the time axis over all P+1 frames to obtain the spatiotemporal joint robust reference background value to replace the single-frame median value, which is used to suppress the inter-frame reference fluctuation caused by smoke.

[0009] Furthermore, the step-by-pixel calculation of the temporal change rate and temporal change variance includes: For each pixel in the crosstalk correction image sequence, the absolute value of the inter-frame gray level difference is calculated. The mean value of each pixel position on P inter-frame differences is calculated along the time axis as the temporal change rate. The sum of squares of the differences between the inter-frame difference and the mean value at each pixel position is calculated along the time axis and divided by the number of inter-frame differences as the temporal change variance. The generated crosstalk previously affected the region mask, including: For each frame of saturated pixel mask, dilation is performed along the row and column directions respectively. The expansion width in the row direction is three times the reciprocal of the row attenuation constant, and the expansion width in the column direction is three times the reciprocal of the column attenuation constant. This expansion width corresponds to the distance when the crosstalk attenuation function amplitude decays to five percent of the initial amplitude. The multi-frame masks after dilation are subjected to a pixel-by-pixel logical OR operation along the time axis to generate a mask of the crosstalk-affected region.

[0010] Furthermore, the classification of all image pixels into three categories—moving target region, dynamic smoke region, and stable region—includes: The time-domain rate of change and the time-domain variance were normalized using Z-score standardization. For pixels with a value of zero in the mask of the area affected by crosstalk, they are classified according to the normalized temporal change rate and temporal change variance: pixels with a temporal change rate greater than the rate high threshold and a temporal change variance greater than the variance high threshold are marked as moving target areas; pixels with a temporal change rate between the rate low threshold and the rate high threshold and a temporal change variance not greater than the variance low threshold are marked as smoke dynamic areas; pixels with a temporal change rate less than the rate low threshold and a temporal change variance less than the variance low threshold are marked as stable areas. For pixels with a median value of one in the mask of the area affected by crosstalk, classification is performed based on the normalized temporal change rate and variance after crosstalk correction, using the same threshold conditions as above. The two classification results are merged to generate a moving target region mask, a smoke dynamic region mask, and a stable region mask covering the entire map; The high rate threshold is the 95th percentile of the normalized temporal rate of change of pixels with a value of zero in the mask of the region affected by crosstalk, and the low rate threshold is the 30th percentile. The high variance threshold and low variance threshold are respectively taken as the corresponding percentiles of the variance distribution.

[0011] Furthermore, the background grayscale estimation performed on the three types of regions includes: For each pixel location in the dynamic smoke region, the gray-level time series of that pixel location in the crosstalk correction image sequence is extracted. The least squares method is used to fit a first-order linear trend model to obtain the gray-level change slope and intercept. Based on the slope, the gray level before the smoke enters is extrapolated along the decreasing frame number direction. The intercept minus the product of the slope and the preset frame number offset is used as the smoke-free background gray-level estimate of that pixel. For pixels in a stable region, if the crosstalk correction gray value of the current frame is higher than the preset minimum normal gray value threshold, then the gray value of the current frame is taken as the background estimate; if the crosstalk correction gray value of the current frame is lower than the minimum normal gray value threshold, then the pixel is marked as a pixel completely covered by dense smoke and assigned the background estimate corresponding to the minimum transmittance. For a pixel in the moving target region, the background estimate of the pixel is generated by using the background estimate of the reference pixels in the neighborhood that belong to the dynamic or stable smoke region, and then normalizing the sum of the weights by the reciprocal of the Euclidean distance between each reference pixel and the current pixel.

[0012] Furthermore, after the combination is the full-image background grayscale estimation map, the current frame image after crosstalk correction is used as the guide image, and the full-image background grayscale estimation map is subjected to guided filtering. The guided filtering establishes a linear relationship between the output and the guide image within a local window centered on each pixel. The linear coefficient is solved by minimizing the difference between the output and the input, so that the output image maintains the gradient direction consistency at the edge of the guide image and suppresses the grayscale jump caused by the difference in estimation methods in different regions at the region boundary.

[0013] Furthermore, the step of substituting the Beer-Lambert law into the inverse solution of local transmittance includes: The attenuation map is generated by taking the absolute value of the difference between the current frame image after crosstalk correction and the grayscale estimation map of the whole background pixel by pixel. Gaussian low-pass filtering is then applied to the attenuation map to generate a smooth attenuation map. The atmospheric scattered light component is estimated using the dark channel prior method. The dark channel prior method takes the minimum gray value of the current frame image after crosstalk correction within a local neighborhood window centered on each pixel as the estimated value of the atmospheric scattered light component at that pixel. The local transmittance at each pixel is calculated as a quotient of the difference between the smooth attenuation map, the estimated grayscale map of the entire image, and the atmospheric scattered light component. A range constraint is applied to the transmittance estimation map, limiting the transmittance to a closed interval between a preset minimum transmittance threshold and one, thus generating a constrained transmittance estimation map.

[0014] Furthermore, after generating the jointly denoised image, the process also includes: Based on the constrained transmittance estimation map and saturation source location information, a partitioned dynamic range mapping enhancement is performed on the jointly denoised image to generate a denoised and detail-enhanced infrared image, including: For pixels in the vicinity of the saturated source within a preset proximity distance from the boundary of any saturated connected region, logarithmic compression mapping is performed, and the gray values ​​of the joint denoised image are normalized to the full scale range of the detector after taking the natural logarithm. For low-transmittance smoke region pixels with constrained transmittance below the low transmittance threshold, nonlinear gamma correction enhancement is performed. The gamma coefficient is determined by linear interpolation between the minimum gamma coefficient and the maximum gamma coefficient based on the constrained transmittance at that pixel. The smaller the constrained transmittance, the smaller the gamma coefficient and the stronger the enhancement. For normal region pixels that are not in the vicinity of the saturation source and whose transmittance after constraint is not lower than the low transmittance threshold, perform piecewise linear mapping to linearly scale the grayscale value to the output grayscale range. For pixels within the boundary transition zone of the three types of regions, a linear weighted fusion is performed on the mapping results of the two sides of the region, using the inverse ratio of the distance from the pixel to the boundary of the two adjacent regions as the weight, to eliminate gray-level jumps at the boundary of the region.

[0015] This invention provides an infrared imaging image denoising and detail enhancement system for harsh environments, comprising: The image sequence acquisition module is used to acquire the current frame and the previous P frames of raw infrared images continuously acquired by the infrared focal plane array detector, forming a short-time image sequence. The saturation detection module is used to detect saturated pixels in each frame of a short-time image sequence and perform connected component labeling to generate saturation source location information and saturated pixel mask for each frame. The crosstalk correction module is used to extract the unsaturated pixel grayscale profiles along the row and column directions for each saturated connected component in each frame of the image, fit the exponential decay model to obtain the crosstalk attenuation parameters, estimate and subtract the electronic crosstalk component from the original image, and generate a crosstalk corrected image sequence. The temporal feature analysis module is used to calculate the temporal change rate and temporal change variance of the crosstalk correction image sequence pixel by pixel, and to generate a crosstalk-affected region mask by performing a pixel-by-pixel logical OR operation on the time axis after expanding the multi-frame saturated pixel mask along the row and column directions. The region classification module is used to classify the pixels of the entire image into three categories: moving target region, dynamic smoke region, and stable region based on the temporal change rate, temporal change variance, and crosstalk-affected region mask. The background estimation module is used to perform background grayscale estimation on the three types of regions based on the classification results of the three types of regions and the crosstalk correction image sequence, and combine them into a background grayscale estimation map of the whole image. The transmittance estimation module is used to substitute the crosstalk-corrected current frame image and the full-image background grayscale estimation map into the Beer-Lambert law to solve the local transmittance and generate a constrained transmittance estimation map after range constraint. The descattering recovery module is used to perform descattering recovery operations on the current frame image after crosstalk correction using the constrained transmittance estimation map to generate a jointly denoised image.

[0016] This invention employs a collaborative processing sequence of prioritizing crosstalk removal followed by temporal analysis and transmittance estimation of smoke. This solves the technical problem of denoising failure caused by mutual interference when the two degradation mechanisms of strong radiation source electronic crosstalk and persistent smoke spatially overlap. The invention achieves the following technical effects: By independently performing electronic crosstalk estimation and subtraction based on exponential decay fitting of row and column grayscale profiles for each frame, the pulse-like grayscale abrupt changes generated by crosstalk fringes in the temporal domain are eliminated. This allows subsequent region classification based on temporal variation features to obtain the real scene change patterns of the crosstalk-affected areas, avoiding misclassification of smoke regions caused by crosstalk artifacts. By constructing a mask of the crosstalk-affected areas and reclassifying the pixels within it based on the crosstalk-corrected temporal features, the integrity of the three types of region classification masks is ensured. By performing temporal grayscale trend extrapolation estimation of the smoke-free background grayscale in dynamic smoke regions, the dependence on pre-acquired smoke-free background reference frames is eliminated, enabling transmittance estimation to be performed normally in scenarios with persistent smoke coverage. Combined with partitioned dynamic range mapping enhancement, effective recovery of target detail information simultaneously obscured by electronic crosstalk noise and smoke in infrared images of harsh environments is achieved. Attached Figure Description

[0017] Figure 1 This is a flowchart of an infrared imaging image denoising and detail enhancement method for harsh environments provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the trend of representative pixel grayscale value changes in a short-time image sequence provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the row-directed electronic crosstalk exponential decay fitting curve provided in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the temporal variation features of representative pixels provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of representative pixel transmittance estimation results provided in an embodiment of the present invention; Figure 6 This is a schematic diagram comparing grayscale values ​​before and after descattering recovery according to an embodiment of the present invention; Figure 7 This is a schematic diagram comparing grayscale values ​​before and after partition enhancement according to an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the relationship between the gamma coefficient and transmittance in the low transmittance region provided in an embodiment of the present invention. Detailed Implementation

[0018] In infrared imaging applications in harsh environments, infrared focal plane array detectors need to image composite degradation scenarios where extremely strong radiation sources coexist with aerosols such as smoke or dust within the field of view. Such scenarios typically occur in applications such as continuous explosion battlefield situational awareness monitoring and industrial high-temperature furnace area monitoring. Strong radiation sources cause detector pixel saturation, and electronic crosstalk produces striped artifacts in the row and column directions adjacent to the saturated region; continuously released smoke causes overall or localized image contrast degradation.

[0019] Existing infrared image denoising methods typically handle the two types of degradation mechanisms mentioned above independently: for electronic crosstalk stripe noise, a radial exponential decay halo model centered on the centroid of the saturated region is used for fitting and subtraction; for contrast attenuation caused by smoke, a pre-acquired smoke-free background reference frame is used, and the transmittance is solved by substituting the pixel-by-pixel difference between the current degraded image and the smoke-free background reference frame into the Beer-Lambert law, and then descattering restoration and detail enhancement are performed.

[0020] However, when strong radiation source electronic crosstalk and persistent smoke coexist in the field of view and partially overlap spatially, the aforementioned independent processing methods face coupling failure. Specifically, the following three technical problems exist: First, electronic crosstalk generates pulse-like gray-level abrupt changes in the rows and columns adjacent to the saturation region. These abrupt changes manifest as drastic changes in the time domain that are synchronized with the appearance or disappearance of the saturation source. Smoke region classification algorithms based on time-domain variation characteristics misclassify crosstalk-affected pixels as moving targets, thus excluding them from the smoke transmittance estimation range and causing large-area holes in the smoke estimation within the crosstalk-covered area.

[0021] Second, smoke attenuation causes the grayscale baseline of neighboring pixels in the saturation region to shift non-uniformly with space and time. The reference background values ​​at the far end of the row and column on which the crosstalk attenuation parameter fitting depends are no longer stable, resulting in a systematic bias.

[0022] Third, in scenarios with continuous smoke release, it is impossible to obtain an effective smoke-free background reference frame, and the transmittance estimation method that relies on the smoke-free background reference frame becomes completely ineffective.

[0023] The mutual interference between the two degradation mechanisms mentioned above leads to a severe decrease in denoising accuracy and failure to recover details.

[0024] According to an embodiment of this invention, a method for denoising and enhancing details in infrared imaging images for use in harsh environments is provided. It should be understood that this method for combined crosstalk and smoke denoising and detail enhancement is executed by an image processing unit in an infrared imaging system equipped with an infrared focal plane array detector. This image processing unit has the capability to perform real-time data processing on the continuously output digital infrared image frames from the detector.

[0025] At least one embodiment of the present invention discloses a method for denoising and enhancing details in infrared imaging images for use in harsh environments, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain a short-time image sequence of the original infrared image; Acquire the current frame and its preceding frames continuously acquired by the infrared focal plane array detector. Frame infrared raw image, constituting a collection Short-time image sequence of frames ,in For the current frame, For the first Frame image, The frame number, The range of values ​​is to , This represents the number of historical frames required for time-domain analysis.

[0026] It should be noted that, The value of is a positive integer that satisfies the stability of the time-domain statistical characteristics, for example, it can be a preset value between 5 and 20. The selection needs to balance the reliability of time-domain statistics with the timeliness of scene changes. When the value is too small, the time-domain statistics are easily affected by instantaneous noise. When the value is too large, the slow-changing scene changes will be averaged out.

[0027] Step 2: Detect saturated pixels and generate saturation source location information; For short-time image sequences In each frame of the image, the grayscale value is compared pixel by pixel with the detector's full-scale value. To achieve grayscale values The pixels are labeled as saturated pixels. Connectivity labeling is performed on all saturated pixels in each frame. The centroid coordinates and bounding rectangles of each saturated connected component are extracted to generate a set of saturation source location information for each frame. and saturated pixel mask .

[0028] It should be noted that the above saturation pixel determination condition is that the grayscale value equals... This means that the detector outputs digital grayscale values ​​that reach the quantization limit, which is a direct characteristic of pixel response saturation caused by strong radiation sources. Connectivity component labeling uses the 8-connectivity neighborhood criterion, grouping spatially adjacent saturated pixels into the same saturated connected component.

[0029] Step 3: Estimate and subtract electronic crosstalk components to generate a crosstalk-corrected image sequence; For each frame in a short-time image sequence For each saturated connected component, extract the unsaturated pixel grayscale sequences in the same row and column as the saturated connected component, calculate the crosstalk attenuation model parameters in the row and column directions, and generate the electronic crosstalk component estimation map of the saturated source; then, superimpose the electronic crosstalk component estimation maps of all saturated sources in the frame pixel by pixel to generate the total crosstalk estimation map of the frame. From the original image Subtract from each pixel Negative values ​​are then cropped to zero to generate a crosstalk-corrected image. Perform the above processing on each frame to generate a crosstalk-corrected image sequence. .

[0030] Specifically, the calculation process of the crosstalk attenuation model parameters mentioned above includes the following sub-steps: Step 301: For a saturated connected component, extract all unsaturated pixel gray values ​​in the same row as the center row of the saturated connected component to form a row-oriented gray-level profile sequence; extract all unsaturated pixel gray values ​​in the same column as the center column of the saturated connected component to form a column-oriented gray-level profile sequence.

[0031] Step 302: In the row-oriented grayscale profile sequence, select elements whose distance from the boundaries of all saturated connected components in the frame exceeds a preset safety distance. The pixel grayscale values ​​are used to calculate the midpoint, which is then used as the row-oriented robust baseline background value. Perform the same operation on the column-oriented grayscale profile sequence to obtain the column-oriented robust reference background value. .in This is the minimum pixel spacing that is farthest from the boundaries of all saturated connected components in the row or column direction, used to eliminate interference from near-end pixels still affected by crosstalk in the calculation of the baseline background value. The value can be obtained by pre-calibrating based on the crosstalk characteristics of the detector, for example, by taking a distance of 50 to 100 pixels.

[0032] Step 303: Along the row direction, calculate the gray values ​​of each unsaturated pixel from near to far from the boundary of the saturated connected region. The differences constitute a row-wise difference sequence. ,in The distance between the pixel and the boundary of the saturated region; for Fitting the exponential decay function: in, For the row difference sequence at distance The difference at the point, The initial amplitude of the horizontal crosstalk. Let be the row-direction attenuation constant. This represents the pixel distance from the boundary of the saturation region along the row direction. Represents an exponential function. Along the column... Perform the same fitting to obtain the initial magnitude of column crosstalk. and column attenuation constant The parameters of the aforementioned exponential decay function are fitted using the least squares method. The input is the distance-difference pairs of unsaturated pixels, and the output is... , (or , ).

[0033] It should be noted that, and All are dimensionless grayscale differences. Pixel distance (unit: pixels) The unit is the reciprocal of pixels. and Since all quantities are in the same dimension, namely grayscale units, the quantities in the above exponential decay formula are consistent and can be directly calculated.

[0034] Step 304: Calculate the position of each unsaturated pixel based on the row and column crosstalk attenuation parameters. The row-direction crosstalk estimates and column-direction crosstalk estimates are then superimposed pixel-by-pixel to generate an electronic crosstalk component estimation map of the saturated source. Specifically, for each pixel position... Let its distance from the boundary of its saturated connected region along the direction be . The distance along the column direction is The estimated value of the crosstalk is then... The estimated value of the column crosstalk is The sum of these two values ​​is the estimated electronic crosstalk component at that pixel. This represents an exponential function.

[0035] It should be noted that in step 302, the median statistic is used instead of the mean to calculate the row-wise robust baseline background value. and column-oriented robust baseline background value This is because the median has the ability to resist individual abnormal gray values ​​that may exist in the row and column directions. When there are residual effects from other saturation sources in the row or column, the median statistic can avoid extreme values ​​from deviating from the benchmark.

[0036] In this embodiment of the application, in order to address the non-uniform shift of the reference background value at the far end of the row and column due to smoke attenuation, in step 302, the grayscale profile sequence of the row direction that is more than the distance from all saturation regions is further adjusted. The far-end pixels along the time axis Obtain the spatiotemporal joint robust baseline background value by taking the temporal median of the frame. Replace single frame median This processing utilizes the gradual variation in smoke grayscale decay and further suppresses inter-frame baseline fluctuations caused by smoke through temporal median filtering, making the baseline for fitting the crosstalk attenuation parameter more stable.

[0037] Step 4: Calculate the temporal variation characteristics of the image sequence after crosstalk correction, and generate a mask for the crosstalk-affected region; Crosstalk correction image sequence The absolute value of the grayscale difference between adjacent frames is calculated pixel by pixel. Calculate the position of each pixel along the time axis exist The mean value over the inter-frame differences is used to generate a temporal rate of change map. Calculate the variance of the inter-frame difference for each pixel location along the time axis to generate a temporal variance map. .

[0038] Furthermore, the time-domain rate of change plot The specific calculation formula is as follows: in, pixel position The rate of change in the time domain at that point, To correct crosstalk in the image sequence of the first Frame at pixel position grayscale value at that location The frame number, The range of values ​​is to , This represents the number of historical frames in a short-time image sequence.

[0039] Time-domain variation variance plot The specific calculation formula is as follows: in, pixel position The variance of the time-domain variation at that point. In the above two equations... Each inter-frame difference corresponds to a frame sequence number pair , from Take to ,common Item. In both of the above formulas, each term represents a grayscale unit. The unit is gray level. The unit is the square of the gray level.

[0040] Simultaneously, for all frame saturation pixel masks generated in step 2 First, apply a saturated pixel mask to each frame. Perform dilation along the row and column directions, with the expansion width matching the distance at which the crosstalk attenuation function amplitude decays to 5% of its initial amplitude in step 3. Then, perform a pixel-by-pixel logical OR operation along the time axis to generate a mask for the crosstalk-affected region. Crosstalk has affected the regional mask. A pixel with a median value of 1 indicates that it was marked as a saturated pixel or was located within the crosstalk range of the row and column containing the saturated connected component in any frame of a short-time image sequence.

[0041] Furthermore, the aforementioned extended width is determined by the attenuation constant of the crosstalk attenuation function: taking the row direction as an example, when the estimated crosstalk value in the row direction... decay to initial amplitude When it is 5%, the corresponding distance is ,Right now The width is expanded in the row direction using this distance; the same applies to the column direction. The width is expanded as a column-direction expansion. Represents an exponential function. This represents the natural logarithm function. The method for determining this extended width ensures that the masking of the area previously affected by crosstalk is protected. It can fully cover the pixel range where the crosstalk amplitude is still 5% higher than the initial amplitude.

[0042] Step 5: Perform region classification based on temporal variation characteristics and crosstalk-affected region masks; Based on the time-domain rate of change graph Time-domain variation variance plot Crosstalk has affected the area mask The system performs region classification on each pixel location in the image and generates a corrected three-class region classification mask.

[0043] Before performing region classification, the time-domain rate of change map Plot of variance with time domain variation Z-score standardization was used to normalize the data to eliminate the influence of the difference in dimensions on the threshold comparison. The normalized time-domain rate of change map and time-domain variance map were used for subsequent classification.

[0044] Specifically, the region classification process includes the following sub-steps: Step 501: For The pixels, based on the normalized time-domain rate of change map Plot of variance with time domain variation Categorize: Will and The pixels are marked as moving target regions; and The pixels are marked as the dynamic smoke region; and The pixels marked are stable regions. , , , This is the preset classification threshold.

[0045] Furthermore, the above classification criteria cover the temporal variation patterns of three typical pixel types: pixels in moving target regions exhibit high change rates and high variance due to target movement; pixels in dynamic smoke regions exhibit medium change rates and low variance due to slow smoke diffusion; and pixels in stable regions exhibit low change rates and low variance due to a static background. For pixels that do not meet any of the above classification criteria, they are assigned to the category closest to their temporal change rate to ensure that all pixels in the image are classified.

[0046] Step 502: For Since the crosstalk component has been removed in step 3, the temporal variation characteristics of this type of pixel in the crosstalk-corrected image sequence have returned to the real scene variation law. Therefore, the same threshold condition as in step 501 is used to re-perform the classification judgment based on the normalized temporal variation rate and variance after crosstalk correction, and the classification result of this type of pixel is generated.

[0047] Step 503: Merge the classification results of steps 501 and 502 to generate a corrected three-class region classification mask covering the entire image, including a moving target region mask. Smoke dynamic area mask and stable region mask .

[0048] In this embodiment of the application, in order to adapt the selection of the classification threshold to different scenario conditions, the above-mentioned classification threshold... , , , based on The normalized temporal rate of change and variance statistical distribution of pixels are adaptively determined: Take the 95th percentile of the temporal change rate of this portion of pixels. Take the 30th percentile. and Take the corresponding percentile of the variance distribution.

[0049] Step 6: Estimate the grayscale of the smoke-free background in the entire image; Based on the corrected three-class region classification mask and crosstalk correction image sequence Background grayscale estimation was performed on the three types of regions separately, and the estimation results of all regions were combined into a background grayscale estimation map of the entire image. .

[0050] Specifically, the background grayscale estimation process includes the following sub-steps: Step 601: Dynamic area of ​​smoke Each pixel position in Extract the pixel location in the crosstalk correction image sequence Gray-scale time series A first-order linear trend model is fitted to the gray-scale time series: in, pixel position In the The grayscale value of the frame, The slope of the grayscale change. The intercept is... The frame number, The range of values ​​is to , Corresponding to the current frame, This corresponds to the earliest historical frame. The parameters of the above first-order linear trend model are fitted using the least squares method, with the input being a sequence pair of frame number and corresponding pixel grayscale value. The output is the slope. and intercept Based on the slope obtained from the fitting Along frame number Extrapolate the decreasing direction to the gray level before the smoke enters, and estimate the smoke-free background gray value of that pixel. Specifically, let the extrapolation offset of the frame number corresponding to the smoke entry point be... The estimated grayscale value for a smokeless background is: in pixel position Estimated grayscale value of the smokeless background at the location. The variable is a positive integer, and is set to a fixed frame offset to stabilize the extrapolation result. This is used when the starting frame of the smoke cannot be clearly determined in the short-time image sequence. Use a fixed preset value.

[0051] It should be noted that the frame number in the above formula The direction of the value is defined as Corresponding to the current frame, This corresponds to the earliest historical frame, meaning the frame number increases positively along the timeline. As smoke continues to enter the scene, the pixel grayscale affected by smoke attenuation changes accordingly. The decrease (i.e., the progression of time) shows a downward trend, corresponding to the slope. ;along Decrease direction outward, i.e. Take the negative offset to extrapolate the results The gray level is higher than the current observed gray level, corresponding to a higher background gray level before the smoke entered, and the extrapolation direction is consistent with the physical meaning.

[0052] Furthermore, in the above first-order linear trend model, , and All are grayscale units. The frame number is a dimensionless frame number. The unit is grayscale levels per frame. The unit is a frame, therefore the extrapolation is... The unit is gray level, and Dimensional consistency, extrapolation results The unit is gray level, and all dimensions are consistent, so calculations can be performed directly.

[0053] Step 602: For the stable region The pixels in the current frame, if the crosstalk correction grayscale value Higher than the preset minimum normal grayscale threshold Then, the grayscale value of the current frame is directly taken as the background estimate. ;like If the pixel is completely obscured by dense smoke, it will be marked as a pixel with the lowest transmittance. The corresponding background estimate. Wherein It serves as the lower bound of the normal grayscale range in a stable region, used to determine whether a pixel has been severely obscured by dense smoke and cannot be represented by its current grayscale value.

[0054] Step 603: Target the moving area For each pixel in the image, a background estimate is generated using distance-weighted spatial interpolation, utilizing the background estimates of pixels within the dynamic or stable smoke region of that pixel's neighborhood. Specifically, let the number of reference pixels participating in the interpolation within the neighborhood be [number missing]. indivual, For the number of reference pixels, the first The coordinates of the reference pixels are Background estimated value The reference pixel and the current pixel The Euclidean distance is , The reference pixel number, The range of values ​​is to Then the estimated background value of the current pixel is: in, For the current moving target area pixels Background estimate at the location, For the first Background estimate at each reference pixel For the first The Euclidean distance between each reference pixel and the current pixel. This represents the total number of reference pixels in the neighborhood. The background estimates of the reference pixels are weighted and summed using the reciprocal of the Euclidean distance between each reference pixel, and then normalized. The input consists of the coordinates and background estimates of pixels in the dynamic and stable smoke regions within the neighborhood. The output is the background estimate of pixels in the current moving target region.

[0055] Step 604: Combine the background grayscale estimates of the three types of regions from steps 601 to 603 into a full-image background grayscale estimate. .

[0056] Furthermore, to improve the spatial smoothness of the overall background grayscale estimation image, after step 604, the current frame image after crosstalk correction is used. As a guide image, the grayscale estimation map of the entire background is performed. Guided filtering is performed to suppress gray-level jumps at region boundaries caused by differences in estimation methods across different regions, while preserving the image edge structure. The input to the guided filter is the estimated gray-level image of the entire background. and guide image The output is a grayscale estimate of the entire background image with smoothed edges.

[0057] Furthermore, guided filtering is an edge-preserving filtering method based on a local linear model of the guided image: it estimates the background grayscale of the entire image. Within a local window centered on each pixel, establish the output and guide images (i.e., The linear relationship between the input and output is solved by minimizing the difference between the input and output, so that the output image maintains the gradient direction consistency at the edge of the guide image, thereby eliminating abrupt changes while preserving the edge structure in the smooth region.

[0058] Step 7: Estimate local transmittance based on Beer-Lambert's law; The current frame image after crosstalk correction Compared with the grayscale estimation image of the whole image background Calculate the absolute value of the difference pixel by pixel to generate an attenuation map. ; for attenuation diagram Perform Gaussian low-pass filtering to generate a smooth attenuation map. It is used to suppress spatial high-frequency noise generated by pixel-by-pixel differential, so that the transmittance estimation has reasonable smooth transition characteristics in space.

[0059] Smooth attenuation map Substituting the infrared band form of Beer-Lambert's law, we can solve for the local transmittance value at each pixel location. According to Beer-Lambert's law, the attenuation relationship of infrared radiation passing through scattering media such as smoke is as follows: in, For the current frame image after crosstalk correction, in pixels grayscale value at that location Estimating the grayscale of the background image in pixels. The value at that location, For pixels Local transmittance at that location This represents the atmospheric scattered light component. Based on the smooth attenuation map. And the full image background grayscale estimation image Inverse solution Specifically, the above formula can be rearranged to obtain: This generates a transmittance estimation map. The atmospheric scattered light component The dark channel prior method is used to approximate the estimation by the minimum gray value in the local neighborhood of the image.

[0060] Furthermore, the specific implementation of the dark channel prior method is as follows: for each pixel... Within a local neighborhood window centered on the image, retrieve the current frame image after crosstalk correction. The minimum grayscale value of all pixels within this window is taken as the atmospheric scattered light component at that pixel. The estimated value is based on the physical principle that most local areas in natural scenes contain dark pixels with grayscale close to zero, and the grayscale value of these dark pixels approximately reflects the contribution of atmospheric scattered light.

[0061] It should be noted that the smooth attenuation graph in the above formula... Full image background grayscale estimation image and atmospheric scattered light component All are grayscale units, molecular smooth attenuation map With denominator With the same dimensions, dividing yields the dimensionless transmittance. All quantities have the same dimensions, so calculations can be performed directly.

[0062] Transmittance estimation diagram Apply range constraints to the transmittance estimation map. Limited to Within the range, of which A preset minimum transmittance threshold is used to prevent division by zero or excessive amplification in areas obscured by dense smoke, generating a constrained transmittance estimation map. .

[0063] Furthermore, the physical meaning of the above range constraint is: transmittance This represents the proportion of energy retained after infrared radiation passes through a smoke medium, and its physical value range is... ,in Indicates that the medium is completely transparent. This indicates that the medium is completely opaque. Set the lower bound to... The non-zero value is to ensure the denominator of the descattering recovery operation in step 8. The value is not zero to avoid numerical calculation overflow; the upper bound is set to 1 because the transmittance cannot exceed the physical upper limit of 1. The value can be preset based on the maximum optical thickness of the smoke in the scene, for example, a preset value between 0.1 and 0.2.

[0064] Step 8: Perform descattering recovery operation to generate a jointly denoised image; Using constrained transmittance estimation plot The current frame image after crosstalk correction Perform descattering recovery operation: in, The grayscale values ​​of the image after descattering recovery. For the current frame image after crosstalk correction, in pixels grayscale value at that location These are estimates of the atmospheric scattered light components. For constrained transmittance estimation maps at pixels The value at that location. This operation is performed on each pixel location, generating a jointly denoised image for crosstalk and smoke. .

[0065] It should be noted that the current frame image after crosstalk correction in the numerator of the above formula... and All values ​​are in grayscale units; transmittance estimation diagram after denominator constraint. Since it is a dimensionless transmittance, the numerator and denominator have the same dimensions, and it can be directly calculated and output. The unit is gray level, which is the same as the gray dimension of the input image, and can be directly used as input for subsequent enhancement processing.

[0066] In this embodiment of the application, in order to further recover the image denoised by the joint denoising of crosstalk and smoke... To remove target details obscured by noise and smoke, the following steps are also included: Step 9: Perform partitioned dynamic range mapping enhancement to generate the final denoised and detail-enhanced infrared image; Denoising images with crosstalk and smoke Based on the constrained transmittance estimation map Using the saturation source location information generated in step 2, perform partitioned dynamic range mapping enhancement to generate the final denoised and detail-enhanced infrared image. .

[0067] Specifically, the partition mapping enhancement process includes the following sub-steps: Step 901: Based on the saturation source location information, determine the saturation source's neighboring region within a preset proximity distance from any saturation connected component boundary. Perform logarithmic compression mapping on the pixels within the saturation source's neighboring region: in, To output the image in pixels grayscale value at that location To denoise the image by combining crosstalk and smoke at the pixel level grayscale value at that location Denoising images for crosstalk and smoke The maximum gray value in the vicinity of the medium-saturation source. This is the detector's full-scale value. This represents the natural logarithm function. This mapping compresses the high dynamic range gray values ​​near the saturation source, allowing target details in the vicinity of the saturation source, which are obscured by strong radiation, to be displayed within a finite number of gray levels.

[0068] Step 902: For The low transmittance smoke region pixels, among which To achieve a low transmittance threshold, nonlinear gamma correction enhancement is performed: in, To output the image in pixels grayscale value at that location To denoise the image by combining crosstalk and smoke at the pixel level grayscale value at that location This is the detector's full-scale value. The gamma coefficient is related to transmittance. The value of and the transmittance estimation diagram after constraint Positive correlation, i.e., the transmittance estimation plot after constraint. The smaller The smaller the value, the greater the enhancement. Specifically, Determined by the following formula: in, This represents the minimum gamma coefficient corresponding to a transmittance approaching zero. For transmittance to approach The maximum gamma coefficient corresponding to time, For constrained transmittance estimation maps at pixels The values ​​at each location are preset constants and satisfy the following conditions: This results in a smaller gamma coefficient and a stronger nonlinear enhancement when the transmittance is lower.

[0069] It should be noted that in the mapping formulas of steps 901 and 902, the image is denoised jointly by crosstalk and smoke. and All are grayscale units. This is a dimensionless ratio with consistent dimensions for all components. The output is... The unit is gray level, which is the same as the dimension of the input image.

[0070] Step 903: For regions not adjacent to the saturation source and For pixels in the normal region, perform piecewise linear mapping to linearly scale the grayscale values ​​to the output grayscale range. Specifically, let the minimum grayscale value in the normal region be... The maximum grayscale value is The output will be: in, To output the image in pixels grayscale value at that location To denoise the image by combining crosstalk and smoke at the pixel level grayscale value at that location This represents the minimum grayscale value within the normal area. This represents the maximum grayscale value within the normal area. This is the detector's full-scale value. That is, the dynamic range of grayscale in the normal region is linearly stretched to... The output range.

[0071] Step 904: For the boundaries of the above three types of regions, calculate the weighted fusion coefficient based on the distance between each pixel and the boundary of each region and the transmittance value at that pixel. Perform gradient fusion on the mapping results of adjacent regions to eliminate gray-level jumps at the region boundaries and generate the final denoised and detail-enhanced infrared image. Specifically, for pixels located within the transition zone between two adjacent regions... Let its distance from the boundary of region A be . The distance from the boundary of region B is The mapping results of regions A and B at this pixel are respectively and The fused output is: in, To output the image after fusion at the pixel level grayscale value at that location For pixels Distance from the boundary of region A, For pixels Distance from the boundary of region B, This is the mapping result of region A at this pixel. This is the mapping result of region B at this pixel. That is, the mapping results on both sides are linearly weighted and fused using the inverse ratio of the distance from the pixel to the boundary of each region as the weight. The closer to the boundary of a region, the greater the weight of the mapping result of that region, thereby achieving a smooth transition at the boundary of the region.

[0072] The crosstalk and smoke joint denoising and detail enhancement method provided in this embodiment addresses the problem of spatial overlap and mutual interference between the two degradation mechanisms of strong radiation source electronic crosstalk and continuous smoke in harsh environments. It adopts a collaborative processing order of prioritizing crosstalk removal before performing smoke time-domain analysis and transmittance estimation, thus achieving effective decoupling and joint compensation of the two coupled degradation mechanisms.

[0073] Specifically, because this implementation method first performs electronic crosstalk estimation and subtraction based on row and column grayscale profile exponential decay fitting on each frame in step 3, eliminating the pulse-like grayscale abrupt changes generated by crosstalk stripes in the time domain, the region classification based on time domain change characteristics in steps 4 and 5 can obtain the real scene change law of the crosstalk-affected area, and solve the coupling interference problem of misclassification of smoke area caused by crosstalk artifacts.

[0074] Because in this implementation, step 4 generates a mask for the region affected by crosstalk by taking the union of multiple frames of saturated pixel masks along the time axis. In step 5, the mask for the area affected by crosstalk is applied. The pixels within the area are reclassified based on the temporal characteristics after crosstalk correction, thus avoiding interference from historical crosstalk residues in the determination of moving targets and smoke areas, and ensuring the integrity and accuracy of the three-category region classification mask after correction.

[0075] Because in this embodiment, step 6 is based on the crosstalk correction image sequence By performing temporal grayscale trend extrapolation on the dynamic smoke region to estimate the grayscale of the smoke-free background, the dependence on pre-acquired smoke-free background reference frames is eliminated, thus solving the problem of transmittance estimation failure caused by the unavailability of smoke-free background reference frames in continuous smoke coverage scenarios.

[0076] Therefore, this embodiment provides a clean temporal analysis input by first performing crosstalk denoising and then using crosstalk-corrected image sequences to generate accurate transmittance through a collaborative processing flow. Combined with partitioned adaptive mapping enhancement, it can recover target detail information in infrared images of harsh environments that are simultaneously obscured by electronic crosstalk noise and smoke.

[0077] like Figure 2-8 As shown, the method in this embodiment operates as follows: An infrared focal plane array (FLA) detector imaging system is installed in the high-temperature furnace area of ​​a steel plant to continuously monitor the thermal distribution of the furnace and surrounding equipment. The system has an image resolution of 640×512 pixels and a detector full-scale value... (14-bit quantization), frame rate 25 frames / second. During a monitoring mission in 20XX, a high-temperature molten metal radiation source was present at the furnace opening, causing local pixel saturation on the detector. Simultaneously, leaking smoke from the furnace continuously filled the field of view, creating a combined degradation scene of electronic crosstalk and smoke. The image processing unit needs to perform joint denoising and detail enhancement on this combined degradation to restore the thermal details of pipes, valves, and other equipment surrounding the furnace. The system preset parameters are: (Historical frame count) (pixels) , , , .

[0078] The image processing unit reads the current frame from the detector buffer. Together with the previous 10 historical images, they form a short image sequence containing 11 frames. .in For the current frame, These are the earliest historical frames. Each frame has an image size of 640×512 pixels, with a grayscale value ranging from 0 to 16383. Taking a representative pixel in the center region of the image as an example, the grayscale values ​​of each frame are recorded as follows.

[0079] Table 1. Representative pixel grayscale values ​​of short-time image sequences (area around pixel coordinates 320×256): The sequence shows a trend of increasing grayscale value as the frame number increases, reflecting the continuous decay of pixel grayscale caused by smoke over time (the grayscale of the current frame is lower than that of the historical frames).

[0080] Detect saturated pixels pixel by pixel in each frame of the sequence. (Using the current frame...) For example, within the coordinate range of approximately column 480 to 510 and row 220 to 260 at the furnace opening, the pixel grayscale values ​​all reach [value missing]. The pixel is identified as saturated. Eight-connectivity labeling is performed on this connected component, and the centroid coordinates and circumscribed rectangle are extracted to generate a set of saturation source location information for the current frame. and saturated pixel mask .

[0081] Table 2: Location information of saturated connected components in each frame: The positions of saturated connected components in each frame remain stable within a short time series, corresponding to the continuous strong radiation of high-temperature molten metal at the furnace opening.

[0082] With the current frame Taking a saturated connected region (region 1, centroid column 495, row 240) as an example, the parameter estimation process of the crosstalk attenuation model is explained.

[0083] Row processing: Extract all unsaturated pixel grayscale values ​​from row 240, selecting those that are more than [distance] from the boundaries of all saturated connected components. The spatiotemporal joint robust baseline background value is calculated for the far-end pixels (i.e., pixels with column coordinates less than 420 or greater than 570). This is obtained using the temporal midpoint of the far-end pixels across 11 frames. Construct a difference sequence along the row direction. The least squares fit result for the fitted exponential decay function is: , (pixels) ).

[0084] Column processing: Extract the unsaturated pixels in column 495, and similarly obtain... The fitting result is , (pixels) ).

[0085] At the distance from the boundary of the saturation region Taking a pixel as an example, the estimated value of the row crosstalk is: Crosstalk range (distance corresponding to attenuation to 5% of initial amplitude): Generate a total crosstalk estimation map for all saturation sources in the current frame. ,from Subtract from each pixel After cropping the negative values, the crosstalk corrected image is obtained. The same processing is performed on all 11 frames to generate a crosstalk-corrected image sequence. .

[0086] Based on the crosstalk-corrected image sequence, the absolute value of the inter-frame grayscale difference is calculated pixel by pixel, and temporal features are statistically analyzed. Simultaneously, the saturated pixel mask for each frame is expanded by 73 pixels along the row and column directions (taking...). Then, perform a pixel-by-pixel logical OR operation on the 11-frame masks to generate masks for areas previously affected by crosstalk. .

[0087] Taking several representative pixel locations in the image as examples, the calculation results of the temporal change rate and variance are shown. The temporal change rate of pixel (320, 256) is calculated as follows: From the data in Table 1, the grayscale difference between each adjacent frame is approximately 76. (Grayscale).

[0088] Table 3. Temporal variation characteristics and crosstalk mask status of representative pixels: right The pixels are Z-score normalized using the normalized temporal rate of change and variance, based on an adaptive threshold ( Take the 95th percentile, Classify using the 30th percentile. The pixels are re-evaluated based on the temporal characteristics after crosstalk correction.

[0089] Taking each pixel in Table 3 as an example, the classification results are as follows: (This table shows the classification process in step 5. It corresponds to the data in Table 3 and is not a separate result table.) right For the pixels, step 3 has eliminated crosstalk pulse abrupt changes, and its normalized temporal characteristics revert to the dynamic region pattern of smoke. Therefore, it is re-identified as a dynamic region of smoke rather than a moving target region, thus correcting the misjudgment caused by crosstalk.

[0090] Based on the corrected three-class region classification mask, the smokeless background gray level is estimated for each type of region.

[0091] Smoke dynamic region (taking pixel (320, 256) as an example): Extract the grayscale time series of this pixel from the crosstalk correction sequence, and fit a first-order linear trend model to 11 frames of data. From the crosstalk correction data in Table 1, the grayscale of this pixel changes with the frame number. Increase and rise (high grayscale in historical frames), slope (Grayscale level / frame), intercept Take the extrapolated offset. ,but: Stable region (taking pixel (150, 180) as an example): Current frame crosstalk correction grayscale value Take directly .

[0092] Moving target region (taking pixel (490, 310) as an example): Select 3 reference pixels belonging to the dynamic or stable smoke region within its neighborhood, and perform distance-weighted interpolation: Taking reference pixel 1 as coordinates (475, 310), background estimate 8976, and distance 15 pixels; reference pixel 2 as coordinates (490, 295), background estimate 9102, and distance 15 pixels; and reference pixel 3 as coordinates (505, 310), background estimate 8843, and distance 15 pixels, and assuming all three are at equal distances, the weighted result is the average of the three. After combining the estimation results of the three types of regions, with Guided filtering is performed on the guided image to eliminate gray-level jumps at region boundaries, resulting in a gray-level estimation image of the entire background. .

[0093] Taking pixel (320, 256) as an example, the process of transmittance estimation is explained.

[0094] grayscale of the current frame after crosstalk correction Background estimate Attenuation .

[0095] The smoothed attenuation value is obtained after Gaussian low-pass filtering. The atmospheric scattered light component is estimated by the minimum gray value within a local neighborhood. The transmittance is: Table 4. Transmittance estimation results for representative pixels: Original transmittance at pixel (260, 380) After constraint, the transmittance is taken. This prevents division by zero or numerical overflow in subsequent calculations.

[0096] Taking each pixel in Table 4 as an example, perform descattering and restoration operations to obtain a jointly denoised image. Taking pixel (380, 300) as an example (belonging to the smoke-covered area, the transmittance after constraint) ): After descattering recovery, the pixel grayscale was restored from 4512 to 9084, which is close to the background estimate of 9156, indicating that the smoke attenuation was effectively compensated.

[0097] Based on joint denoising image After constraining the transmittance estimation map, perform partitioned dynamic range mapping enhancement.

[0098] Saturation source proximity region (step 901, logarithmic compression mapping): Taking pixel (495, 245) as an example, it is located within 20 pixels of the boundary of the saturated connected region and belongs to the saturation source proximity region. This region is selected. , : Low transmittance smoke region (step 902, gamma correction enhancement): Taking pixel (260, 380) as an example, the transmittance after constraint... Calculate the gamma coefficient: Pick , but (Grayscale), a significant improvement over the original grayscale 1634, restoring the detail brightness of the dense smoke area.

[0099] Normal region (step 903, linear mapping): Taking pixel (150, 180) as an example, the normal region is selected. , , : Region boundary fusion (step 904): For the pixel (310, 370) at the boundary between the low-transmittance smoke region and the normal region, set... (Distance from the boundary of the low transmittance region) (Distance from the normal region boundary), mapping results on both sides , : Table 5 shows the output results of the partition enhancement for representative pixels: The entire processing flow demonstrates a clear data flow logic: First, step 1 acquires 11 frames of raw infrared image sequence from the detector cache, providing a data foundation for all subsequent temporal analyses. Step 2 locates the saturated connected components at the furnace opening in each frame, generating a saturated pixel mask. Step 3, based on a spatiotemporal joint robust benchmark and exponential decay fitting of row and column grayscale profiles, estimates and subtracts crosstalk components frame by frame, converting the original sequence into a crosstalk correction sequence, eliminating the dual interference of smoke attenuation and crosstalk pulses on temporal analysis. Step 4 calculates the temporal rate of change map and variance map on the crosstalk correction sequence, and simultaneously expands the multi-frame saturated masks and takes the union to generate a crosstalk history mask, providing a clean temporal input and complete historical influence markers for region classification in step 5. Step 5 performs classification on pixels inside and outside the crosstalk history mask, correcting smoke region pixels that were originally misclassified as moving targets due to crosstalk, generating three types of region masks covering the entire image. Step 6 employs three strategies—temporal extrapolation, direct value extraction, and spatial interpolation—for the three types of regions respectively, combining them to generate a full-image background grayscale estimation map, thus eliminating the reliance on pre-acquired smokeless reference frames. Step 7 utilizes the background estimation map to solve for transmittance using the Beer-Lambert law, and ensures numerical stability through range constraints. Step 8 performs descattering recovery, restoring low-grayscale pixels after smoke attenuation to the background grayscale level. Step 9 performs logarithmic compression, gamma correction, or linear mapping based on region type and transmittance magnitude, and gradually blends the regions at their boundaries, ultimately generating a denoised and enhanced infrared image with clearly discernible thermal details of pipes, valves, and other equipment around the furnace.

[0100] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A method for denoising and enhancing details in infrared imaging images for use in harsh environments, characterized in that, Includes the following steps: The current frame and the previous P frames of raw infrared images continuously acquired by the infrared focal plane array detector are used to form a short-time image sequence. For each frame in a short-time image sequence, saturated pixels are detected and connected component labeling is performed to generate saturation source location information and saturated pixel mask for each frame; For each saturated connected component in each frame of the image, unsaturated pixel grayscale profiles are extracted along the row and column directions, respectively. An exponential decay model is fitted to obtain crosstalk attenuation parameters. Electronic crosstalk components are estimated and subtracted from the original image to generate a crosstalk-corrected image sequence. Calculate the temporal change rate and temporal change variance pixel by pixel for the crosstalk correction image sequence; expand the multi-frame saturated pixel mask along the row and column directions and perform a pixel-by-pixel logical OR operation along the time axis to generate a mask for the crosstalk-affected region. Based on the temporal change rate, temporal change variance, and crosstalk-affected area mask, the pixels of the entire image are classified into three categories: moving target area, smoke dynamic area, and stable area. Based on the classification results of the three types of regions and the crosstalk correction image sequence, background gray level estimation is performed on the three types of regions respectively and combined into a full-image background gray level estimation map; The process of performing background grayscale estimation on the three types of regions includes: For each pixel location in the dynamic smoke region, the gray-level time series of that pixel location in the crosstalk correction image sequence is extracted. The least squares method is used to fit a first-order linear trend model to obtain the gray-level change slope and intercept. Based on the slope, the gray level before the smoke enters is extrapolated along the decreasing frame number direction. The intercept minus the product of the slope and the preset frame number offset is used as the smoke-free background gray-level estimate of that pixel. For pixels in a stable region, if the crosstalk correction gray value of the current frame is higher than the preset minimum normal gray value threshold, then the gray value of the current frame is taken as the background estimate; if the crosstalk correction gray value of the current frame is lower than the minimum normal gray value threshold, then the pixel is marked as a pixel completely covered by dense smoke and assigned the background estimate corresponding to the minimum transmittance. For a pixel in the moving target region, the background estimate of the pixel is generated by using the background estimate of the reference pixels in the neighborhood that belong to the dynamic or stable smoke region, and then normalizing the sum of the weights of the Euclidean distances between each reference pixel and the current pixel. Substitute the crosstalk-corrected current frame image and the full image background grayscale estimation map into the Beer-Lambert law to solve for the local transmittance, and generate a constrained transmittance estimation map after range constraint. The constrained transmittance estimation map is used to perform descattering recovery operation on the current frame image after crosstalk correction, generating a joint denoised image.

2. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 1, characterized in that, The fitted exponential decay model obtains crosstalk attenuation parameters, estimates and subtracts electronic crosstalk components from the original image, including: For a saturated connected component, extract all unsaturated pixel gray values ​​in the same row as the center row of the saturated connected component to form a row-oriented gray-level profile sequence, and extract all unsaturated pixel gray values ​​in the same column as the center column of the saturated connected component to form a column-oriented gray-level profile sequence. In the row-oriented grayscale profile sequence, select the pixel grayscale values ​​that are more than a preset safe distance from the boundaries of all saturated connected components in the frame, and calculate the midpoint as the row-oriented robust reference background value; perform the same operation on the column-oriented grayscale profile sequence to obtain the column-oriented robust reference background value; The difference between the grayscale value of each unsaturated pixel and the robust baseline background value along the row direction is calculated to form a row difference sequence. The row difference sequence is fitted with an exponential decay function using the least squares method. The exponential decay function means that the difference is equal to the initial amplitude of the row crosstalk multiplied by an exponential function with a negative exponent of the product of the row decay constant and the distance of the pixel from the boundary of the saturated region. The initial amplitude of the row crosstalk and the row decay constant are obtained. The same fitting is performed along the column direction to obtain the initial amplitude of the column crosstalk and the column decay constant. Based on the row and column crosstalk attenuation parameters, the row and column crosstalk estimates for each unsaturated pixel are calculated and superimposed pixel by pixel to generate the electronic crosstalk component estimation map for the saturated source. The electronic crosstalk component estimation maps for all saturated sources in the frame are superimposed pixel by pixel to generate the total crosstalk estimation map for the corresponding frame. The total crosstalk estimation map is subtracted pixel by pixel from the original image and negative values ​​are cropped to zero to generate the crosstalk correction image.

3. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 2, characterized in that, In the calculation of the row-oriented robust reference background value and the column-oriented robust reference background value, the median value of the far-end pixels in the row-oriented grayscale profile sequence that are more than a preset safe distance from all saturated regions is taken along the time axis on all P+1 frames. The spatiotemporal joint robust reference background value is obtained to replace the single-frame median value and is used to suppress the inter-frame reference fluctuation caused by smoke.

4. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 1, characterized in that, The pixel-by-pixel calculation of the temporal change rate and temporal change variance includes: For each pixel in the crosstalk correction image sequence, the absolute value of the inter-frame gray level difference is calculated. The mean value of each pixel position on P inter-frame differences is calculated along the time axis as the temporal change rate. The sum of squares of the differences between the inter-frame difference and the mean value at each pixel position is calculated along the time axis and divided by the number of inter-frame differences as the temporal change variance. The generated crosstalk previously affected the region mask, including: For each frame of saturated pixel mask, dilation is performed along the row and column directions respectively. The expansion width in the row direction is three times the reciprocal of the row attenuation constant, and the expansion width in the column direction is three times the reciprocal of the column attenuation constant. This expansion width corresponds to the distance when the crosstalk attenuation function amplitude decays to five percent of the initial amplitude. The multi-frame masks after dilation are subjected to a pixel-by-pixel logical OR operation along the time axis to generate a mask of the crosstalk-affected region.

5. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 4, characterized in that, The classification of all image pixels into three categories—moving target region, dynamic smoke region, and stable region—includes: The time-domain rate of change and the time-domain variance were normalized using Z-score standardization. For pixels with a value of zero in the mask of the area affected by crosstalk, they are classified according to the normalized temporal change rate and temporal change variance: pixels with a temporal change rate greater than the rate high threshold and a temporal change variance greater than the variance high threshold are marked as moving target areas; pixels with a temporal change rate between the rate low threshold and the rate high threshold and a temporal change variance not greater than the variance low threshold are marked as smoke dynamic areas; pixels with a temporal change rate less than the rate low threshold and a temporal change variance less than the variance low threshold are marked as stable areas. For pixels with a median value of one in the mask of the area affected by crosstalk, classification is performed based on the normalized temporal change rate and variance after crosstalk correction, using the same threshold conditions as above. The two classification results are merged to generate a moving target region mask, a smoke dynamic region mask, and a stable region mask covering the entire map; The high rate threshold is the 95th percentile of the normalized temporal rate of change of pixels with a value of zero in the mask of the region affected by crosstalk, and the low rate threshold is the 30th percentile. The high variance threshold and low variance threshold are respectively taken as the corresponding percentiles of the variance distribution.

6. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 1, characterized in that, After the combination is the full-image background grayscale estimation map, the current frame image after crosstalk correction is used as the guide image. Guided filtering is performed on the full-image background grayscale estimation map. The guided filtering establishes a linear relationship between the output and the guide image within a local window centered on each pixel. The linear coefficient is solved by minimizing the difference between the output and the input, so that the output image maintains the gradient direction consistency at the edge of the guide image and suppresses the grayscale jump caused by the difference in estimation methods in different regions at the region boundary.

7. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 1, characterized in that, The inverse solution of local transmittance by substituting into Beer-Lambert's law includes: The attenuation map is generated by taking the absolute value of the difference between the current frame image after crosstalk correction and the grayscale estimation map of the whole background. Gaussian low-pass filtering is then applied to the attenuation map to generate a smooth attenuation map. The atmospheric scattered light component is estimated using the dark channel prior method. The dark channel prior method takes the minimum gray value of the current frame image after crosstalk correction within a local neighborhood window centered on each pixel as the estimated value of the atmospheric scattered light component at that pixel. The local transmittance at each pixel is calculated as a quotient of the difference between the smooth attenuation map, the estimated grayscale map of the entire image, and the atmospheric scattered light component. A range constraint is applied to the transmittance estimation map, limiting the transmittance to a closed interval between a preset minimum transmittance threshold and one, thus generating a constrained transmittance estimation map.

8. The method for denoising and enhancing details of infrared imaging images in harsh environments according to claim 1, characterized in that, After generating the jointly denoised image, the following steps are also included: Based on the constrained transmittance estimation map and saturation source location information, a partitioned dynamic range mapping enhancement is performed on the jointly denoised image to generate a denoised and detail-enhanced infrared image, including: For pixels in the vicinity of the saturated source within a preset proximity distance from the boundary of any saturated connected region, logarithmic compression mapping is performed, and the gray values ​​of the joint denoised image are normalized to the full scale range of the detector after taking the natural logarithm. For low-transmittance smoke region pixels with constrained transmittance below the low transmittance threshold, nonlinear gamma correction enhancement is performed. The gamma coefficient is determined by linear interpolation between the minimum gamma coefficient and the maximum gamma coefficient based on the constrained transmittance at that pixel. The smaller the constrained transmittance, the smaller the gamma coefficient and the stronger the enhancement. For normal region pixels that are not in the vicinity of the saturation source and whose transmittance after constraint is not lower than the low transmittance threshold, perform piecewise linear mapping to linearly scale the grayscale value to the output grayscale range. For pixels within the boundary transition zone of the three types of regions, a linear weighted fusion is performed on the mapping results of the two sides of the region, using the inverse ratio of the distance from the pixel to the boundary of the two adjacent regions as the weight, to eliminate gray-level jumps at the boundary of the region.

9. A system for denoising and enhancing details of infrared imaging images in harsh environments, used to perform the method for denoising and enhancing details of infrared imaging images in harsh environments as described in any one of claims 1 to 8, characterized in that, include: The image sequence acquisition module is used to acquire the current frame and the previous P frames of raw infrared images continuously acquired by the infrared focal plane array detector, forming a short-time image sequence. The saturation detection module is used to detect saturated pixels in each frame of a short-time image sequence and perform connected component labeling to generate saturation source location information and saturated pixel mask for each frame. The crosstalk correction module is used to extract the unsaturated pixel grayscale profiles along the row and column directions for each saturated connected component in each frame of the image, fit the exponential decay model to obtain the crosstalk attenuation parameters, estimate and subtract the electronic crosstalk component from the original image, and generate a crosstalk corrected image sequence. The temporal feature analysis module is used to calculate the temporal change rate and temporal change variance of the crosstalk correction image sequence pixel by pixel, and to generate a crosstalk-affected region mask by performing a pixel-by-pixel logical OR operation on the time axis after expanding the multi-frame saturated pixel mask along the row and column directions. The region classification module is used to classify the pixels of the entire image into three categories: moving target region, dynamic smoke region, and stable region based on the temporal change rate, temporal change variance, and crosstalk-affected region mask. The background estimation module is used to perform background grayscale estimation on the three types of regions based on the classification results of the three types of regions and the crosstalk correction image sequence, and combine them into a background grayscale estimation map of the whole image. The transmittance estimation module is used to substitute the crosstalk-corrected current frame image and the full-image background grayscale estimation map into the Beer-Lambert law to solve the local transmittance and generate a constrained transmittance estimation map after range constraint. The descattering recovery module is used to perform descattering recovery operations on the current frame image after crosstalk correction using the constrained transmittance estimation map to generate a jointly denoised image.

Images