A BTB-based image correction method, system, device and medium

Abstract

The application provides a BTB-based image correction method, system, device and medium, and belongs to the technical field of image processing. The method comprises the following steps: converting an original image into a gray image; performing weighted image filtering processing on a current image to be processed to obtain a filtered image; performing down-sampling on the filtered image to obtain a down-sampled image; performing BTB curved surface fitting on the down-sampled image to generate a fitted curved surface, and restoring the fitted curved surface to the original size of the gray image to obtain a target curved surface; generating a current correction image according to the current image to be processed, the target curved surface and a coefficient matrix; calculating a quality evaluation index of the current correction image after each iteration, and terminating the iteration when the increment of the quality evaluation index is less than a preset threshold or decreases; performing post-enhancement processing on an intermediate correction image obtained in the last iteration before the termination of the iteration to obtain a final correction image. The application efficiently removes strong light / thermal radiation of an image and improves the image quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an image correction method, system, device, and medium based on BTB. Background Technology

[0002] When high-speed aircraft fly within the atmosphere, the intense friction with the air creates high-temperature, high-pressure gas currents around the aircraft's surface, especially around the optical radome. The optical radome's temperature rises sharply due to aerodynamic heating, generating intense infrared radiation. This high-temperature gas and the radome's thermal radiation severely interfere with optical imaging and detection systems, leading to problems such as large areas of intense light coverage, reduced contrast, and obscured target details in the image, seriously affecting the accuracy of subsequent target identification and detection.

[0003] To address the aforementioned problem of thermal radiation interference in aero-optics, various correction methods have been proposed in existing technologies. Physical methods mitigate temperature rise and reduce thermal radiation by optimizing the material, thickness, and thermal conductivity of the optical hood. However, these methods suffer from drawbacks such as increased weight, high cost, increased system complexity, and insufficient durability. At the image processing level, methods have attempted to remove thermal radiation interference through filtering, image enhancement, or surface fitting. Among these, the BTB surface fitting method has been successfully applied in 3D modeling visualization and industrial design and manufacturing fields due to its ability to ensure fitting accuracy and continuity.

[0004] However, existing image correction methods still have significant shortcomings. On the one hand, physical methods are costly to implement and slow to respond, making them difficult to meet practical engineering needs. On the other hand, most existing image processing methods fail to fully utilize the C2 continuity and energy minimization advantages of BTB surface fitting, resulting in insufficient accuracy of thermal radiation surface fitting and residual significant thermal radiation effects in the corrected image. Furthermore, existing methods lack adaptive adjustment capabilities when dealing with thermal radiation interference of different intensities, and the contrast and edge sharpness of the corrected image often fail to achieve ideal results, making it difficult to directly use for subsequent high-precision target recognition tasks. Summary of the Invention

[0005] In view of this, it is necessary to provide an image correction method, system, device and medium based on BTB to solve the technical problems of insufficient thermal radiation correction accuracy and obvious residue in the prior art.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides an image correction method based on BTB, comprising: Acquire the original image affected by thermal radiation interference, and convert the original image into a grayscale image; A weighted image filtering process is performed on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image. The filtered image is downsampled to obtain a downsampled image; The BTB surface fitting is performed on the downsampled image to generate a fitted surface, and the fitted surface is restored to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The current corrected image is generated based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, the fitting difference intensity, and the current iteration number of the current image to be processed. The corrected image is used as the current image to be processed in the next iteration. The above filtering, downsampling, surface fitting and correction steps are repeated. After each iteration, the quality evaluation index of the corrected image is calculated. The iteration is terminated when the increment of the quality evaluation index is less than the preset threshold or when it decreases. The intermediate corrected image obtained from the most recent iteration before the iteration ends is subjected to post-enhancement processing to obtain the final corrected image.

[0007] In one possible implementation, the step of performing weighted image filtering on the current image to be processed to obtain a filtered image includes: The current image to be processed is subjected to weighted guided filtering and scale-aware filtering respectively to obtain a first filtered image and a second filtered image; The current image to be processed is normalized to obtain a normalized grayscale image; The first weight and the second weight are calculated based on the gradient magnitudes of the first filtered image and the second filtered image, and the normalized grayscale image. The sum of the first weight and the second weight is 1. The filtered image is obtained based on the first filtered image, the first weight, the second filtered image, and the second weight.

[0008] In one possible implementation, downsampling the filtered image to obtain a downsampled image includes: The filtered image is divided into multiple sampling blocks, and the average gradient magnitude within each sampling block is calculated. If the average gradient magnitude of the current sampling block is less than a preset gradient threshold, the pixel value at a specified position in the current sampling block is determined as the downsampling result of the current sampling block. If the average gradient magnitude of the current sampling block is greater than or equal to a preset gradient threshold, a normalized weight is calculated based on the gradient magnitude of each pixel in the current sampling block, and a weighted average is performed on all pixel values ​​in the current sampling block based on the normalized weight to obtain the downsampling result of the current sampling block. Based on the downsampling results of all the sampling blocks, the downsampled image is obtained by combining them according to the spatial arrangement order of each sampling block in the filtered image.

[0009] In one possible implementation, the surface fitting of each image block of the downsampled image based on the thin-plate spline interpolation algorithm includes: The downsampled image is divided into multiple image blocks, and the Gaussian weights of the corresponding pixels in the image block are calculated based on the standard deviation of the Gaussian kernel, the center point of the Gaussian weight function, and the pixels in the image block. The weighted pixel value is calculated based on the Gaussian weight and the pixel value of the corresponding pixel. A thin-plate spline interpolation function is constructed based on the Gaussian weights, the radial basis function of the thin-plate spline interpolation algorithm, and the coordinates of the control points and query points in the image blocks; the control points and the query points are different pixels in the image blocks. Construct interpolation constraints such that the calculated value of the thin plate spline interpolation function at each control point is equal to the weighted pixel value of the control point; Construct zero-sum constraints such that the algebraic sum of the weights of all radial basis functions in the thin plate spline interpolation function is zero, and the algebraic sum of the products of the weights of all radial basis functions and their coordinates is zero; Based on the thin plate spline interpolation function, the interpolation results satisfying the interpolation constraints and the zero-sum constraints within the image block are solved, and the local fitted surface of the image block is generated according to the interpolation results.

[0010] In one possible implementation, the step of stitching and fusing the boundaries of adjacent image blocks using a biquadratic Bézier surface interpolation algorithm to obtain the fitted surface includes: For two adjacent image blocks, the function values ​​of the local fitted surfaces of the two image blocks on the adjacent boundaries are extracted and used as Bezier boundary control values. Based on the Bézier boundary control value, a biquadratic Bézier surface interpolation function is constructed to calculate the Bézier fitting value at each pixel position within the boundary region of two adjacent image blocks. The pixel values ​​of two adjacent image blocks within the boundary region are updated based on the Bezier fitting value, and all image blocks are combined in spatial order to obtain a fitted surface corresponding to the downsampled image size.

[0011] In one possible implementation, the step of generating the corrected image based on the current image to be processed, the target surface, and the coefficient matrix includes: Calculate the gradient magnitude at each pixel position in the current image to be processed, and normalize the gradient magnitude based on the maximum value of the gradient magnitude to obtain a smooth hotspot mask; The difference between the current image to be processed and the target surface is calculated to obtain a residual image. The residual image is normalized and then transformed to obtain a residual weight matrix. The dynamic decay factor is calculated based on the ratio of the current iteration number to the maximum iteration number. The coefficient matrix is ​​calculated based on the smoothed hot zone mask, the residual weight matrix, and the dynamic attenuation factor. Multiply the target surface by the coefficient value at the corresponding position in the coefficient matrix to obtain the weighted target surface; The current image to be processed is subtracted pixel by pixel from the weighted target surface to obtain the current corrected image.

[0012] One possible implementation also includes: Using the intermediate corrected image as the input image, an objective function is constructed. The objective function is used to constrain the enhanced image to be close to the intermediate corrected image, while applying sparse constraints to the gradient of the enhanced image. The regularization weights of the sparse constraints are positively correlated with the gradient magnitude. The objective function is solved using an optimization algorithm to obtain the first enhanced image; Calculate the average gray value of the first enhanced image, determine the gamma correction parameters based on the average gray value, and perform gamma transformation on the first enhanced image based on the gamma correction parameters to obtain the second enhanced image; The second enhanced image is sorted by pixel value, and pixel values ​​that are above a first preset threshold and below a second preset threshold are removed to obtain the final corrected image; the first preset threshold is greater than the second preset threshold.

[0013] Secondly, the present invention also provides a BTB-based image correction system, comprising: A preprocessing module is used to acquire the original image affected by thermal radiation interference and convert the original image into a grayscale image; The weighted filtering module is used to perform weighted image filtering on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image; The downsampling module is used to downsample the filtered image to obtain a downsampled image; The fitting module is used to perform BTB surface fitting on the downsampled image to generate a fitted surface, and restore the fitted surface to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The correction module is used to generate the corrected image based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, fitting difference intensity, and current iteration number of the current image to be processed. The iterative processing module is used to take the current corrected image as the current image to be processed in the next iteration, repeat the above filtering, downsampling, surface fitting and correction steps, calculate the quality evaluation index of the current corrected image after each iteration, and terminate the iteration when the increment of the quality evaluation index is less than a preset threshold or decreases; perform post-enhancement processing on the intermediate corrected image obtained in the most recent iteration before the termination of the iteration to obtain the final corrected image.

[0014] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the BTB-based image correction method described in any of the above implementations.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps of the BTB-based image correction method described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: The image correction method, system, device, and medium based on BTB provided by this invention firstly construct a complete and efficient thermal radiation interference removal process by combining weighted filtering, adaptive downsampling, BTB surface fitting, coefficient matrix weighted correction, and an iterative early stopping mechanism. Furthermore, by using a BTB fitting method based on thin plate splines and biquadratic Bézier surfaces, both accurate fitting of local thermal radiation and smooth transition of block boundaries can be simultaneously considered, effectively avoiding the block effect or fitting deviation problems commonly found in traditional surface fitting. Furthermore, by introducing a coefficient matrix and an iterative early stopping mechanism, the correction intensity can be adaptively adjusted according to the residual thermal radiation in the current image, and the iteration can be terminated in a timely manner when the quality index no longer improves, ensuring the correction effect while avoiding unnecessary computational overhead. Furthermore, by using post-enhancement processing as the final step, the contrast and edge sharpness of the image can be further improved on the basis of effectively removing thermal radiation interference, resulting in a high-quality, high-contrast, and sharp-edge final corrected image, making the corrected image more in line with human visual needs or the requirements of subsequent target recognition tasks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an embodiment of the BTB-based image correction method provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 6 For the present invention Figure 1 A schematic diagram of an embodiment of S105; Figure 7 This is a schematic flowchart of another embodiment of the BTB-based image correction method provided by the present invention; Figure 8Figure 1 shows the BTB surface fitting results, where (a) is the original thermal radiation degradation image, (b) is the image of (a) after weighted image filtering, (c) is the image of (b) after downsampling, and (d) is the three-dimensional image of (c) after BTB surface fitting. Figure 9 The images show the changes during the iterative correction process, where (a) is the filtered image of the first iteration, (b) is the filtered image of the second iteration, (c) is the filtered image of the third iteration, and (d) is the filtered image of the fourth iteration. Figure 10 The images show the correction results with a reference image, where (a) is the original thermal radiation degradation image, (b) is the corrected image with the best results achieved through iteration, (c) is the clear reference image in the same scene, and (d) is the curve of SSIM and PSNR changing with the number of iterations. Figure 11 Figure 1 shows the effect of implementing this method for aero-optical thermal radiation correction when a reference image is available. Figure 2 shows the thermal radiation degradation image (with a clear image), Figure 3 shows the correction image with the best effect achieved through iteration, Figure 4 shows the actual clear image, and Figure 5 shows the change curves of SSIM and PSNR during the iteration process. Figure 12 Figure 1 shows the effect of aero-optical thermal radiation correction when no reference image is available. Figure 2 shows the thermal radiation degradation image (no clear image is available), Figure 3 shows the correction image with the best effect achieved by iteration, Figure 4 shows the image after image enhancement, and Figure 5 shows the change curves of SNR and EOG during the iteration process. Figure 13 A schematic diagram of an embodiment of the BTB-based image correction system provided by the present invention; Figure 14 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] Before demonstrating the embodiments, the following terms will be explained.

[0024] BTB (Bezier and Thin Plate Spline (based) Image Correction Method) is an algorithm that combines Bézier surface and Thin Plate Spline (TPS) algorithms.

[0025] This invention provides an image correction method, system, device, and medium based on BTB, which are described below.

[0026] Figure 1 This is a schematic flowchart of an embodiment of the BTB-based image correction method provided by the present invention, as shown below. Figure 1 As shown, the BTB-based image correction method includes: S101. Acquire the original image affected by thermal radiation interference and convert the original image into a grayscale image.

[0027] It should be noted that: First, an original image affected by thermal radiation interference is acquired. This original image can be an image acquired by an infrared imaging system in a high-temperature environment, such as an image acquired after the optical radome of a high-speed aircraft is aerodynamically heated. Subsequently, this original image is converted into a grayscale image to reduce the computational complexity of subsequent processing and eliminate redundant information between color channels, providing a unified brightness input basis for subsequent filtering and surface fitting.

[0028] S102. Perform weighted image filtering on the current image to be processed to obtain the filtered image; in the first iteration, the current image to be processed is the grayscale image.

[0029] It should be noted that in the current iteration, a weighted image filtering operation is performed on the current image to be processed to obtain the filtered image. In the first iteration, the current image to be processed is the grayscale image mentioned above. The purpose of weighted image filtering is to suppress high-frequency noise while preserving the overall structural information of the image, making the thermal radiation component more prominent, which facilitates subsequent surface fitting to accurately capture the spatial distribution of thermal radiation.

[0030] S103. The filtered image is downsampled to obtain a downsampled image.

[0031] It should be noted that the filtered image is downsampled to obtain a downsampled image with reduced resolution. By reducing the image resolution, the computational load in the subsequent BTB surface fitting process can be significantly reduced, improving the algorithm's efficiency, while preserving the main low-frequency characteristics of the thermal radiation field.

[0032] S104. Perform BTB surface fitting on the downsampled image to generate a fitted surface, and restore the fitted surface to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface.

[0033] It should be noted that BTB surface fitting is performed on the downsampled image to generate the corresponding fitted surface. Specifically, the BTB surface fitting includes two stages: First, based on the thin-plate spline interpolation algorithm, surface fitting is performed on each image block divided from the downsampled image. Thin-plate spline interpolation has energy minimization and C... 2 The continuous characteristic allows for precise matching of the thermal radiation intensity distribution within each image block. Then, a biquadratic Bézier surface interpolation algorithm is used to stitch and fuse the boundaries between adjacent image blocks, resulting in a complete fitted surface. The Bézier surface interpolation flexibly adjusts the stitching transition through control points, ensuring continuity and smoothness at block boundaries. After the above fitting is completed, the size of the fitted surface is restored to the same original size as the grayscale image through interpolation, obtaining the target surface. This target surface represents the spatial distribution estimate of thermal radiation interference in the current image to be processed.

[0034] S105. Based on the current image to be processed, the target surface, and the coefficient matrix, a correction image is generated. The coefficient matrix is ​​generated based on the residual thermal radiation distribution, fitting difference intensity, and current iteration number of the current image to be processed.

[0035] It should be noted that: based on the current image to be processed, the aforementioned target surface, and a dynamically generated coefficient matrix, an image correction operation is performed to generate the corrected image for this iteration. Specifically, the correction method involves subtracting the pointwise product of the target surface and the coefficient matrix from the current image to be processed. The coefficient matrix is ​​generated based on the residual thermal radiation distribution in the current image to be processed, the intensity of the fitting difference, and the current iteration number. By dynamically adjusting the coefficient matrix, the intensity of thermal radiation removal can be adaptively controlled at different iteration stages, avoiding overcorrection or undercorrection.

[0036] S106. The corrected image is used as the current image to be processed in the next iteration. The above filtering, downsampling, surface fitting and correction steps are repeated. After each iteration, the quality evaluation index of the corrected image is calculated. The iteration is terminated when the increment of the quality evaluation index is less than the preset threshold or when it decreases.

[0037] It should be noted that the corrected image generated in this iteration is used as the current image to be processed in the next iteration, and the filtering, downsampling, surface fitting, and correction steps S102 to S105 are repeated. After each iteration, a quality evaluation index for the corrected image is calculated. This quality evaluation index can be based on the structural similarity (SSIM) or peak signal-to-noise ratio (PSNR) of the reference image, or it can be a contrast or gradient energy index without a reference image. The iteration process terminates when the increment of the quality evaluation index is less than a preset threshold (indicating that the correction effect has stabilized) or when the index decreases (indicating that correction has occurred).

[0038] S107. Perform post-enhancement processing on the intermediate corrected image obtained in the most recent iteration before the termination of the iteration to obtain the final corrected image.

[0039] It should be noted that: the corrected image generated in the most recent iteration before the iteration terminates is obtained, and post-enhancement processing is performed on it to obtain the final corrected image. The purpose of post-enhancement processing is to further improve the contrast and edge sharpness of the image, thereby improving the visual effect.

[0040] In summary, the image correction method based on BTB provided in this invention firstly constructs a complete and efficient thermal radiation interference removal process by combining weighted filtering, adaptive downsampling, BTB surface fitting, coefficient matrix weighted correction, and an iterative early stopping mechanism. Furthermore, by using a BTB fitting method based on thin plate splines and biquadratic Bézier surfaces, it can simultaneously ensure accurate fitting of local thermal radiation and smooth transition of block boundaries, effectively avoiding the block effect or fitting bias problems commonly found in traditional surface fitting. Furthermore, by introducing a coefficient matrix and an iterative early stopping mechanism, the correction intensity can be adaptively adjusted according to the residual thermal radiation in the current image, and the iteration can be terminated in a timely manner when the quality index no longer improves, ensuring the correction effect while avoiding unnecessary computational overhead. Furthermore, by using post-enhancement processing as the final step, the image contrast and edge sharpness can be further improved on the basis of effectively removing thermal radiation interference, resulting in a high-quality, high-contrast, and sharp-edge final corrected image, making the corrected image more suitable for human visual needs or the requirements of subsequent target recognition tasks.

[0041] In some embodiments of the present invention, such as Figure 2 As shown, step S102 includes: S201. Perform weighted guided filtering and scale-aware filtering on the current image to be processed to obtain a first filtered image and a second filtered image.

[0042] It should be noted that two different filtering operations are performed on the current image to be processed. Specifically, a weighted guided filter is applied to the image to obtain the first filtered image; simultaneously, a scale-aware filter is applied to the image to obtain the second filtered image. The weighted guided filter can smooth flat areas while preserving edges, while the scale-aware filter can adaptively preserve or suppress image details at different scales.

[0043] S202. Normalize the current image to be processed to obtain a normalized grayscale image.

[0044] It should be noted that the current image to be processed is normalized to obtain a normalized grayscale image. This normalization process maps pixel values ​​to a uniform numerical range, eliminating the influence of differences in exposure or dynamic range between different images, and providing a unified scale benchmark for the stable calculation of subsequent weights.

[0045] S203. Calculate the first weight and the second weight based on the gradient magnitudes of the first filtered image and the second filtered image, and the normalized grayscale image. The sum of the first weight and the second weight is 1.

[0046] It should be noted that the first weight and the second weight are calculated based on the gradient magnitude information of the first and second filtered images, combined with the aforementioned normalized grayscale image. The first weight corresponds to the fusion coefficient of the first filtered image (weighted guided filtering result), and the second weight corresponds to the fusion coefficient of the second filtered image (scale-aware filtering result), with the sum of the first and second weights being 1. This calculation process utilizes the characteristic that gradient magnitude reflects the local texture complexity and edge strength of the image, allowing the weight allocation to be adaptively adjusted according to the image content.

[0047] S204. Based on the first filtered image, the first weight, the second filtered image, and the second weight, the filtered image is obtained.

[0048] It should be noted that: based on the first filtered image and its corresponding first weight, and the second filtered image and its corresponding second weight, a weighted sum is performed to obtain the final filtered image. This fusion process integrates the advantages of the two filters, enabling the output image to obtain filtering effects suitable for the characteristics of different regions. In summary, grayscale and double-precision processing is performed on the degraded image (i.e., the original image of this invention), the same processing is performed on the clear image, and weighted image filtering is performed on the degraded image I. The essence of weighted image filtering is weighted fusion based on the brightness and gradient information of the two filtered images. The two filtering methods are weighted guided filtering and scale-aware filtering, and the corresponding image generation formulas are: ; in, These are the weights and the filter graph for the guided filter, respectively. Here are the weights and guide graph for scale-aware filtering, where .

[0049] The corresponding weights are calculated as follows: , The value is the normalized grayscale value of the corresponding pixel.

[0050]

[0051]

[0052] Where Gx and Gy are the gradients of the filtering result in the x and y directions, respectively. The subscripts of the corresponding formulas change accordingly. In this example, the filter radius of the weighted guided filter is set to 50, and the regularization parameter is 500; the global smoothing weight of the scale-aware filter is 800, the scale parameter is set to 1, the initial penalty weight is 10, the number of IGO iterations is set to 1, the solution tolerance is 1e-5, and the maximum number of PCG iterations is set to 1000.

[0053] In this embodiment, by simultaneously employing two complementary filters—weighted guided filtering and scale-aware filtering—the advantages of each filter can be leveraged for regions with different characteristics in the image. Furthermore, by introducing a dynamic weight allocation mechanism based on gradient magnitude, the fusion ratio of the two filtering results can be adaptively adjusted according to the local texture complexity and edge intensity of the image. This ensures that smooth regions tend to benefit from the suppression effect of scale-aware filtering, while textured edge regions tend to benefit from the edge-preserving effect of guided filtering, thereby achieving better overall filtering quality. Furthermore, by incorporating normalized grayscale images into the weight calculation, the image's brightness distribution information can be used to guide the weight allocation, making the filtering fusion result more consistent with the human eye's visual sensitivity to different brightness regions, thus improving the visual naturalness of the final filtered image.

[0054] In some embodiments of the present invention, such as Figure 3 As shown, step S103 includes: S301. Divide the filtered image into multiple sampling blocks and calculate the average gradient magnitude within each sampling block.

[0055] It should be noted that the filtered image is divided into multiple sampling blocks of the same size, for example, using a block size of 2×2 pixels. For each sampling block, the gradient magnitude of all pixels within that block is calculated, and then the average of these gradient magnitudes is calculated to obtain the average gradient magnitude of the current sampling block. This average gradient magnitude reflects the complexity of the image texture and the strength of the edges within that block.

[0056] S302. If the average gradient magnitude of the current sampling block is less than a preset gradient threshold, determine the pixel value at a specified position in the current sampling block as the downsampling result of the current sampling block.

[0057] It should be noted that for the current sampling block, its average gradient magnitude is compared with a preset gradient threshold. If the average gradient magnitude of the current sampling block is less than the preset gradient threshold, it indicates that the image content within that block is relatively smooth, with little texture variation, and belongs to a simple region. In this case, the pixel value at a specified position (e.g., the top left corner of the block) is directly determined as the downsampling result for that block. This processing method has extremely low computational cost and can quickly complete the downsampling of flat regions.

[0058] S303. If the average gradient magnitude of the current sampling block is greater than or equal to a preset gradient threshold, calculate the normalized weight based on the gradient magnitude of each pixel in the current sampling block, and perform a weighted average of all pixel values ​​in the current sampling block based on the normalized weight to obtain the downsampling result of the current sampling block.

[0059] It should be noted that if the average gradient magnitude of the current sampling block is greater than or equal to a preset gradient threshold, it indicates that the block contains relatively rich texture details or obvious edge structures, belonging to a complex region. In this case, a normalized weight is first calculated based on the gradient magnitude of each pixel within the block, so that pixels with larger gradient magnitudes (i.e., pixels with more obvious texture or edge features) receive higher weight values, and pixels with smaller gradient magnitudes receive lower weight values. Then, a weighted average is calculated based on these normalized weights for all pixel values ​​within the block, and the result of the weighted average is used as the downsampling result for that block. This processing method can preserve important detail information in complex regions during the downsampling process.

[0060] S304. Based on the downsampling results of all the sampling blocks, the sampling blocks are combined according to their spatial arrangement order in the filtered image to obtain the downsampled image.

[0061] It should be noted that after downsampling all sampling blocks, the downsampling results of all blocks are sequentially combined according to their spatial arrangement in the original filtered image (e.g., from left to right, from top to bottom) to form a complete downsampled image. In summary, for the filtered image... For threshold downsampling, assuming the size of the filtered image is M×N, first set the average gradient threshold to 0.3. The gradient threshold is used to determine whether it is a complex region. The sampling block size can be set to 2×2. If the average gradient of a region is less than the average gradient threshold, it is considered a simple region, i.e., the top-left pixel of the region is taken. Then the corresponding sampling calculation formula is: (Equation 1); in, For the downsampled image at position pixel values, This is the pixel value of the first pixel at the top left corner of the current sampling block.

[0062] For complex regions, a weighted average gradient is used, and the corresponding downsampling formula is: (Equation 2); in, For the downsampled image at position pixel values, For the current sampling block, The pixel position within the sampling block. For sampling blocks at pixel positions pixel values, For pixels within the sampling block Normalized weights.

[0063] (Equation 3); in, For pixels within the sampling block Normalized weights, For pixels within the sampling block gradient magnitude, This represents the row and column number of the current sampling block. It is a very small positive number (to prevent division by zero). For the input image in The original pixel value at the location, m and n are the row and column numbers of the sampling block, respectively. In this example, we take m=n=2. The downsampled image is approximately equal to the original image. size.

[0064] In this embodiment, by introducing a region-adaptive downsampling strategy based on average gradient magnitude, differentiated downsampling methods can be adopted according to the different characteristics of local image content, achieving an optimized balance between computational efficiency and information preservation. Furthermore, by using a simple downsampling method with specified pixel values ​​for smooth regions where the average gradient magnitude is less than a threshold, the computational overhead of flat regions can be significantly reduced, improving the overall speed of the downsampling process. Simultaneously, since the information in smooth regions does not change significantly, the simple sampling method does not cause significant information loss. Furthermore, by using a gradient magnitude-weighted averaging downsampling method for complex regions where the average gradient magnitude is greater than or equal to a threshold, pixels with rich textures or significant edges can receive higher contribution weights during downsampling, thereby maximizing the preservation of key detail information while reducing resolution and avoiding blurring or loss of important features due to downsampling.

[0065] In some embodiments of the present invention, such as Figure 4 As shown, step S104 includes: S401. Divide the downsampled image into multiple image blocks, and calculate the Gaussian weights of the corresponding pixels in the image blocks based on the standard deviation of the Gaussian kernel, the center point of the Gaussian weight function, and the pixels within the image blocks.

[0066] It should be noted that the downsampled image is divided into multiple image blocks of equal size. For each image block, based on a pre-set Gaussian kernel standard deviation, using the center point of the image block as the center point of the Gaussian weight function, and combining the coordinate positions of each pixel within the image block, the Gaussian weight corresponding to each pixel in that block is calculated. This Gaussian weight reflects the importance of each pixel in the subsequent surface fitting.

[0067] S402. Calculate the weighted pixel value based on the Gaussian weight and the pixel value of the corresponding pixel.

[0068] It should be noted that the calculated Gaussian weights are multiplied by the original pixel values ​​of the corresponding pixels to obtain the weighted pixel value for each pixel. This weighting process ensures that pixels closer to the block center receive a higher influence in subsequent fitting, while pixels farther away have a correspondingly weaker influence.

[0069] S403. Construct a thin-plate spline interpolation function based on the Gaussian weights, the radial basis function of the thin-plate spline interpolation algorithm, and the coordinates of the control points and query points in the image block; the control points and the query points are different pixels in the image block.

[0070] It should be noted that the thin-plate spline interpolation function is constructed by utilizing the radial basis functions of the thin-plate spline interpolation algorithm and combining the coordinate information of control points and query points in the image block. Here, control points and query points are different pixels in the image block: control points are used to constrain the shape of the interpolation surface, and query points are used to calculate the interpolation result at the position to be fitted. The thin-plate spline interpolation function is composed of a weighted sum of the radial basis functions and the addition of a linear term.

[0071] S404. Construct interpolation constraints such that the calculated value of the thin plate spline interpolation function at each of the control points is equal to the weighted pixel value of the control point.

[0072] It should be noted that the interpolation constraint requires that the calculated value of the thin-plate spline interpolation function at all control points be exactly equal to the weighted pixel value calculated by S402 at that control point. This constraint ensures that the fitted surface accurately meets the weighted numerical requirements at the control point locations.

[0073] S405. Construct zero-sum constraints such that the algebraic sum of the weights of all radial basis functions in the thin plate spline interpolation function is zero, and the algebraic sum of the products of the weights of all radial basis functions and their coordinates is zero.

[0074] It should be noted that constructing zero-sum constraints includes two requirements: first, the algebraic sum of the weights of all radial basis functions in the thin-plate spline interpolation function is zero; second, the algebraic sum of the products of all radial basis function weights and their corresponding control point x-coordinates is zero, and the algebraic sum of their products with their y-coordinates is also zero. These constraints ensure that the coefficients of the linear part of the interpolation function can be uniquely and stably solved.

[0075] S406. Based on the thin plate spline interpolation function, solve the interpolation results within the image block that satisfy the interpolation constraints and the zero-sum constraints, and generate the local fitting surface of the image block according to the interpolation results.

[0076] It should be noted that, based on the thin-plate spline interpolation function constructed above, the interpolation results for all query point positions within the image block are solved under the premise of simultaneously satisfying the interpolation constraint and the zero-sum constraint. Based on the obtained interpolation results, a local fitting surface corresponding to the image block is generated, which accurately describes the spatial distribution of thermal radiation intensity within the image block.

[0077] In summary, for downsampled images Thin-plate spline interpolation (TPS) is performed in blocks. Typically, the number of blocks is odd; here, a 3×3 block is used. The thin-plate spline interpolation function is: (Equation 4); in, For the query point The interpolation result (surface height). The total number of control points. For the first Radial basis function weights for each control point For radial basis functions, For query point To the Control points distance, , , The coefficients of the linear part (affine transformation).

[0078] in, It is the radial basis function of TPS. To query the distance from the control point, For the first Radial basis function weights for each control point , The coefficients are for the linear part, and in order to determine the weights, TPS interpolation requires that it satisfy... (Equation 5); where, For the thin plate spline interpolation function in the th... Calculated values ​​at each control point For the first Weighted pixel values ​​for each control point. And constraints are added. (Formula 6); For the first Radial basis function weights for each control point For the first The coordinates of the control points.

[0079] This constraint ensures that the linear part can be handled correctly. Meanwhile, to enhance the fitting effect, Gaussian weights are introduced: (Equation 7); where, For the first Gaussian weights for each control point For the first The coordinates of the control points The coordinates of the weight center The variance of the Gaussian kernel (1 in this example) is used to control the rate at which the weights decay with distance. (Equation 8); where, These are the weighted control point pixel values. For the first The original pixel values ​​of each control point.

[0080] In this embodiment, a local surface fitting method for image blocks is constructed by combining the thin-plate spline interpolation function with the Gaussian spatial weighting mechanism, which can accurately capture the spatial distribution characteristics of the thermal radiation field. Furthermore, by calculating Gaussian weights with the block center as the weight center and controlling the decay rate with the Gaussian kernel standard deviation, the surface fitting process prioritizes the fitting accuracy of the central region of the block, gradually relaxing constraints towards the outer regions. This effectively suppresses the interference of noise or outliers that may exist at the block edges on the fitting results, improving the robustness and accuracy of the local surface. Furthermore, by introducing Gaussian weights based on the block center as interpolation constraints, the surface fitting process prioritizes the fitting accuracy of the central region, gradually relaxing constraints towards the outer regions. This effectively suppresses the interference of noise or outliers that may exist at the block edges on the fitting results, improving the robustness and accuracy of the local surface. This generates smooth, continuous local surfaces for each image block that conform to the actual distribution of thermal radiation, providing high-quality local fitting results for subsequent block stitching using Bézier surfaces, ensuring the accuracy and stability of the overall BTB surface fitting framework.

[0081] In some embodiments of the present invention, such as Figure 5 As shown, step S104 includes: S501. For two adjacent image blocks, extract the function values ​​of the local fitted surfaces of the two image blocks on the adjacent boundaries, and use them as Bezier boundary control values.

[0082] It should be noted that after obtaining the local fitted surfaces of each image block using the thin-plate spline interpolation algorithm, the boundaries between adjacent image blocks need to be stitched and merged to eliminate stitching marks and form a continuous, smooth, and complete surface. Therefore, for any two adjacent image blocks, this embodiment first determines the position of their common boundary. Since each image block has already generated a local fitted surface through thin-plate spline interpolation, this local fitted surface has a definite function value at each pixel position. In this embodiment, along the common boundary of two adjacent image blocks, the function values ​​at each position on the boundary are extracted from the local fitted surface of the first side (e.g., the left or top side) block, and the corresponding function values ​​are extracted from the local fitted surface of the second side (e.g., the right or bottom side) block. These two sets of extracted function values ​​are used as the Bézier boundary control values ​​for subsequent Bézier surface interpolation, reflecting the surface height of each block at the boundary.

[0083] S502. Based on the Bézier boundary control value, construct a biquadratic Bézier surface interpolation function to calculate the Bézier fitting value at each pixel position within the boundary region of two adjacent image blocks.

[0084] It should be noted that since the two image blocks are fitted with thin-plate splines independently, their function values ​​may differ at the common boundary, resulting in noticeable seams when directly spliced. To eliminate this discontinuity, this embodiment constructs a biquadratic Bézier surface interpolation function based on two sets of extracted Bézier boundary control values. Specifically, for each interpolation position within the boundary region, this embodiment uses the boundary function value from the first side block as the starting control value of the Bézier curve, the boundary function value from the second side block as the ending control value, and the average of the starting and ending control values ​​as the intermediate control value. Based on these three control values, the Bézier fitting value for each pixel position within the boundary region is calculated according to the interpolation rules of quadratic Bézier curves. As the interpolation position moves from the first side boundary to the second side boundary, the Bézier fitting value smoothly transitions from the starting control value to the ending control value, thus achieving a continuous transition in surface height between the two blocks.

[0085] S503. Update the pixel values ​​of two adjacent image blocks within the boundary region according to the Bezier fitting value, and combine all the image blocks in spatial order to obtain a fitted surface corresponding to the size of the downsampled image.

[0086] It should be noted that after obtaining the Bézier fitting values ​​for each pixel position within the boundary region, this embodiment uses these Bézier fitting values ​​as the new pixel values ​​for the boundary region, replacing the original block boundary pixel values. After the above replacement process, the surface height of two adjacent image blocks at the common boundary achieves a consistent transition, and there are no longer abrupt seams. The above boundary extraction, Bézier fitting value calculation, and pixel value update operations are repeated for all adjacent image blocks. After the boundaries of all adjacent blocks have been fused, the updated image blocks are recombined into a complete image according to the original spatial arrangement order of each image block in the downsampled image. Since the boundaries between each block have achieved a smooth transition through Bézier interpolation, the surface obtained after combination is continuous and smooth in all directions. This fitted surface has the same size as the downsampled image, which is the fitted surface output by the BTB surface fitting.

[0087] In summary, the thin-plate spline interpolation function for each 3×3 image block is obtained. Then, the image is traversed through its blocks, and a biquadratic Bézier surface is fitted to the entire image. Finally, the surface is restored to the original image size. The equation of the biquadratic Bézier surface is: (Equation 9); where, For local offset The Bessel fit value at that point, This represents the local offset within an image block, specifically the offset distance of a pixel relative to the boundary of that image block. , The TPS interpolation result (i.e., thin plate spline interpolation result) of two adjacent image blocks (either vertically or horizontally) at the current point. for , The mean. , Take the interpolation results of the TPS image blocks (top, bottom, left, and right) at the current point. Here, is the fitted value, and t is the local offset within the image block, i.e., the pixel offset relative to the boundary of the image block. Control points are generated either above or below the center point, or to the left or right, where the x-direction represents the upper left and lower left image blocks, and the y-direction represents the upper right and lower right image blocks. Finally, interpolation reconstruction is used to restore the surface to the fitted surface of the original image size. .

[0088] In this embodiment, by employing a biquadratic Bézier surface interpolation algorithm to fuse the boundaries of adjacent image blocks, the problem of boundary discontinuities that may occur when stitching together independently fitted blocks can be effectively solved. This seamlessly merges the independently fitted local thin-plate spline surfaces into a single complete fitted surface, achieving a balance between the computational efficiency of block processing and the continuity and integrity of the overall surface. This ensures that subsequent thermal radiation removal operations can be performed based on a spatially smooth, non-abrupt fitted surface, resulting in superior correction performance. Furthermore, by directly using the local fitted surface function values ​​on the boundaries of adjacent blocks as control points for the Bézier surface, the precise alignment of the beginning and end positions of the Bézier transition surface with the original surfaces on both sides can be ensured, avoiding offsets or misalignments caused by improper selection of control points and laying an accurate foundation for a smooth transition. Furthermore, by constructing a biquadratic Bézier surface interpolation function to perform overall interpolation calculations on the boundary region, the natural continuity and curvature consistency of the Bézier surface can be utilized to smoothly transition the boundary region from one side of the segmented surface to the other side of the segmented surface. This eliminates the splicing marks and maintains the morphological characteristics of the original surface near the boundary, without introducing additional distortion or oscillation.

[0089] In some embodiments of the present invention, such as Figure 6 As shown, step S105 includes: S601. Calculate the gradient magnitude of each pixel position in the current image to be processed, and normalize the gradient magnitude according to the maximum value of the gradient magnitude to obtain a smooth hotspot mask.

[0090] It should be noted that during each iteration of correction, the distribution characteristics of residual thermal radiation in the current image to be processed are first analyzed. Since thermal radiation interference typically appears as large, smooth, low-frequency regions in an image, while true image details correspond to regions with large gradient changes, this embodiment distinguishes between smooth hot areas and detail regions by calculating the image gradient. Specifically, the gradient components of each pixel position in the current image to be processed are calculated in the horizontal and vertical directions, and then the gradient magnitude of each pixel position is obtained through square root operations. The gradient magnitude reflects the degree of change in pixel values ​​at that position; the smaller the gradient magnitude, the smoother the region, and the more likely it is to be residual thermal radiation; the larger the gradient magnitude, the richer the texture of the region, and the more likely it is to be true image detail. After obtaining the gradient magnitude of the entire image, the maximum value is found, and the gradient magnitude of each pixel is divided by this maximum value to obtain the normalized gradient magnitude value. Then, the normalized value is subtracted from one to obtain the smooth hot area mask. In a smooth thermal mask, a larger value indicates a smoother region (smaller gradient amplitude), and it is more likely to be a region with residual thermal radiation; a smaller value indicates a richer texture (larger gradient amplitude), and it is more likely to be a region of image detail that needs to be preserved.

[0091] S602. Calculate the difference between the current image to be processed and the target surface to obtain a residual image. Normalize the residual image and then transform it to obtain a residual weight matrix.

[0092] It should be noted that the analysis simultaneously examines the difference between the current image to be processed and the target surface (i.e., the thermal radiation model obtained by fitting the BTB surface). Since the target surface is an estimate of the current thermal radiation distribution, the difference between the two reflects the fitting residual of the thermal radiation model, i.e., the residual thermal radiation that has not yet been removed. The pixel-by-pixel difference between the current image to be processed and the target surface is calculated to obtain the residual image. Larger positive values ​​in the residual image indicate that the current image to be processed is significantly brighter than the target surface at that location, suggesting possible underfitting thermal radiation; larger negative values ​​indicate that the current image to be processed is darker than the target surface at that location, suggesting possible overcorrection. To convert the residual intensity into weight values, the residual image is first normalized to a uniform numerical range, and then the normalized residual intensity is mapped using a nonlinear transformation. This nonlinear transformation uses a sigmoid function, which can compress low-to-medium intensity residual abnormalities to a smaller weight range, while stretching high-intensity residual abnormalities to a larger weight range, thus allowing subsequent correction to focus on areas with strong residual abnormalities.

[0093] S603. Calculate the dynamic decay factor based on the ratio of the current iteration number to the maximum iteration number.

[0094] It should be noted that as the number of iterations increases, the residual thermal radiation in the current image gradually decreases, and the image quality gradually improves. To avoid over-correction in later iterations that could lead to image quality degradation, this embodiment introduces a dynamic attenuation factor to control the trend of the correction intensity with the number of iterations. Specifically, the dynamic attenuation factor is calculated based on the ratio of the current iteration number to the preset maximum iteration number. In the early stages of iteration, this factor has a larger value, allowing for a stronger correction force to quickly remove obvious thermal radiation; as the number of iterations increases, this factor gradually decreases, and the correction force weakens accordingly, only finely adjusting for the residual weak thermal radiation.

[0095] S604. The coefficient matrix is ​​calculated based on the smoothed hot zone mask, the residual weight matrix, and the dynamic attenuation factor.

[0096] It should be noted that the coefficient matrix is ​​generated by integrating information from the above three aspects. For each pixel location in the image, this embodiment determines whether it belongs to a smooth region or a textured region based on the smoothed hotspot mask value at that location. For locations with larger smoothed hotspot mask values ​​(i.e., smooth regions), this embodiment uses a larger base coefficient and superimposes an additional term determined by the residual weight matrix and the dynamic attenuation factor. Since smooth regions are highly likely to be residual thermal radiation, a larger correction coefficient helps to effectively remove interference from these regions. For locations with smaller smoothed hotspot mask values ​​(i.e., textured regions), this embodiment uses a smaller base coefficient and superimposes an additional term determined by the residual weight matrix and the dynamic attenuation factor. Since textured regions contain important image details, a smaller correction coefficient helps to preserve these details and avoid over-correction that causes detail loss. The coefficient matrix generated in the above manner has an independent adaptive correction coefficient for each pixel location. This coefficient simultaneously considers the degree of residual thermal radiation at that location (characterized by the smoothed hotspot mask), the strength of the fitting difference (characterized by the residual weight matrix), and the iteration process (characterized by the dynamic attenuation factor), achieving spatially variable and iteratively adaptive correction strength control.

[0097] S605. Multiply the target surface by the coefficient value at the corresponding position in the coefficient matrix to obtain the weighted target surface.

[0098] It should be noted that after generating the dynamic coefficient matrix, this embodiment multiplies the target surface by the coefficient values ​​at the corresponding positions in the dynamic coefficient matrix to obtain a weighted target surface. This operation results in different correction intensities at different spatial locations: in areas with severe residual thermal radiation, the weighted target surface has larger values, resulting in stronger correction; in areas with rich details, the weighted target surface has smaller values, resulting in weaker correction.

[0099] S606. Perform pixel-by-pixel subtraction between the current image to be processed and the weighted target surface to obtain the current corrected image.

[0100] It should be noted that the corrected image is obtained by subtracting the weighted target surface pixel by pixel from the current image to be processed, and subtracting the weighted thermal radiation model from the original image. The thermal radiation interference in this corrected image has been effectively suppressed, while image details are preserved.

[0101] In summary, the corrected image is obtained by subtracting the degraded image I (i.e., the current image to be processed) from the target surface (i.e., the quadratic Bezier thermal radiation surface) and the corresponding coefficient matrix. The corrected image I_correct is then used as the current image to be processed in the next iteration, and the same process is repeated. Image metrics are calculated during each correction. In this example, if a reference image is available, SSIM and PSNR are calculated; otherwise, SNR and EOG are calculated. When the metric change is less than a certain value (set to 0.001 in this example), it is considered to have achieved the best effect, and correction is stopped early, with the current corrected image considered the final corrected image I_correct_best. Alternatively, correction is stopped early when the metric decreases, with the previous corrected image considered the final corrected image I_correct_best.

[0102] The formula for solving the coefficient matrix mentioned above is: (Equation 10); in, The coefficient matrix is ​​in position The coefficient value at that point is used to control the correction intensity. The residual weight matrix is... As a dynamic decay factor, This is a smooth hotspot mask used to determine whether the current location is a smooth region.

[0103] (Equation 11); in, This is a smoothing hotspot mask value, ranging from 0 to 1. A larger value indicates a smoother region. Current location gradient magnitude, This represents the maximum gradient magnitude in the entire image.

[0104] (Equation 12); in, Let be the gradient component of the image in the horizontal direction (x-direction). This represents the gradient component of the image in the vertical direction (y-direction).

[0105] in (Equation 13); in, This is the residual weight matrix, with values ​​ranging from 0 to 1, which maps the residual strength to nonlinear weights. Given a hyperbolic tangent function, the input is mapped to the interval (-1, 1). Current location The normalized residual intensity (the pixel value of the residual image after normalization), where... This is a dynamic decay factor that decreases linearly from 1 to 0 with the number of iterations. This represents the current iteration number (counting from 0). This represents the maximum number of iterations.

[0106] In this embodiment, a dynamic weighted correction mechanism is constructed by generating a spatially differentiated and iteratively adaptive coefficient matrix by integrating a smoothed hot zone mask, a residual weight matrix, and a dynamic attenuation factor. This mechanism can precisely control the intensity and range of thermal radiation removal. Furthermore, by introducing a smoothed hot zone mask to distinguish between concentrated thermal radiation regions and detail regions in the image, the correction operation can apply a stronger thermal radiation removal force to smooth regions, thereby automatically reducing the removal intensity in detail regions such as texture edges. This effectively removes thermal radiation interference while avoiding excessive weakening or destruction of the image's inherent texture and edges. Furthermore, by incorporating the residual weight matrix into the coefficient matrix generation process, the correction intensity can be dynamically adjusted according to the difference distribution between the current fitted surface and the original image: regions with larger residuals (i.e., regions with inaccurate fitting) receive higher correction weights, thus focusing on insufficiently corrected regions during iteration, accelerating convergence and improving fitting accuracy. Furthermore, by introducing a dynamic attenuation factor that decreases with the number of iterations, the correction process can adopt a strategy from coarse to fine adjustment: the early iterations use larger coefficients to remove thermal radiation significantly and quickly approximate the true thermal radiation distribution; the coefficients gradually decrease in the later iterations to perform fine correction, avoid overcorrection, and ensure stable convergence of the iteration process. It can adaptively adjust the intensity of thermal radiation removal at each step according to the image content, fitting residuals, and iteration progress, achieving accurate, robust, and efficient thermal radiation correction.

[0107] In some embodiments of the present invention, such as Figure 7 As shown, it also includes: S701. Using the intermediate corrected image as the input image, construct an objective function. The objective function is used to constrain the enhanced image to be close to the intermediate corrected image, and at the same time, to impose sparse constraints on the gradient of the enhanced image. The regularization weight of the sparse constraints is positively correlated with the gradient magnitude.

[0108] It should be noted that after iterative correction, post-enhancement processing is performed on the obtained intermediate corrected image to further improve the image's contrast and visual quality. Since thermal radiation in the intermediate corrected image has been largely removed, but issues such as insufficient contrast and unsharp edges may still exist, this embodiment employs a regularization-based image enhancement method. Specifically, this embodiment constructs an objective function containing two mutually constraining constraints. The first constraint requires that the enhanced image remain close to the original intermediate corrected image. This constraint ensures that the enhancement process does not excessively alter the overall structure and content of the image, avoiding distortion or artifacts. If only image sharpness is pursued while ignoring this constraint, the enhancement result may deviate from the real scene, introducing non-existent textures or noise. The second constraint applies a sparsity constraint to the gradient of the enhanced image. The gradient of an image reflects the degree of change in pixel values; locations with larger gradients correspond to image edges and texture details, while locations with smaller gradients correspond to flat areas. The purpose of the sparsity constraint is to encourage the gradient distribution of the enhanced image to be as concentrated as possible, i.e., only a few important edge locations have large gradient values, while the gradient values ​​of most flat areas approach zero. This sparsity helps to highlight the main structure of the image while suppressing minor noise and irrelevant textures. Regularization weights related to the gradient magnitude are introduced into the sparsity constraint.

[0109] Specifically, the regularization weights are positively correlated with the gradient magnitude: larger gradient magnitudes result in larger regularization weights, thus imposing stronger sparsity constraints at those locations and ensuring that the enhanced image retains sharp edges in these areas; conversely, smaller gradient magnitudes result in smaller regularization weights, thus imposing weaker sparsity constraints at those locations and preventing over-smoothing that could lead to detail loss. Through this gradient-weighted design, this embodiment can effectively suppress noise amplification in flat regions while enhancing image edges and details.

[0110] S702. The objective function is solved using an optimization algorithm to obtain the first enhanced image.

[0111] It should be noted that after constructing the objective function, this embodiment uses an optimization algorithm to solve it. Since the objective function contains sparse constraints, traditional gradient descent is difficult to handle directly. This embodiment uses the alternating direction multiplier method. The alternating direction multiplier method is an iterative algorithm suitable for large-scale optimization problems. Its core idea is to decompose the complex original problem into multiple easily solvable subproblems, and gradually approach the optimal solution by alternately updating the variables of each subproblem. The specific solution process is as follows: In this embodiment, the enhanced image in the objective function is used as the first variable, and the horizontal and vertical gradients of the enhanced image are used as the other two auxiliary variables. In each iteration, the auxiliary variables and Lagrange multipliers are first fixed, and the enhanced image is updated. This step is essentially a least squares problem, which can be solved efficiently using fast Fourier transform. Then, the enhanced image and Lagrange multipliers are fixed, and the horizontal and vertical gradients are updated respectively. This step involves L1 norm constraints, which can be solved using soft thresholding. Finally, the Lagrange multipliers are updated to correct the deviation between the first two steps. Through multiple iterations, the algorithm gradually converges to the optimal solution of the objective function. The first enhanced image is obtained after the above solution process. While maintaining an overall similarity to the original intermediate corrected image, the first enhanced image has sharper edges, clearer details, and effectively suppressed noise in flat areas.

[0112] S703. Calculate the average gray value of the first enhanced image, determine the gamma correction parameters based on the average gray value, and perform gamma transformation on the first enhanced image based on the gamma correction parameters to obtain the second enhanced image.

[0113] It should be noted that adaptive gamma correction is performed on the first enhanced image. Since the brightness distribution varies across different images, a fixed gamma parameter cannot be applied to all situations. Therefore, this embodiment adaptively determines the gamma correction parameter based on the average grayscale value of the first enhanced image itself. Specifically, this embodiment first calculates the average grayscale value of all pixels in the first enhanced image. The average grayscale value reflects the overall brightness level of the image: images with higher average grayscale values ​​are generally brighter, while images with lower average grayscale values ​​are generally darker. This embodiment uses 0.5 as the target average brightness (i.e., medium brightness) and calculates the gamma correction parameter based on the relationship between the current average grayscale value and the target average brightness. When the first enhanced image is generally dark (average grayscale value less than 0.5), the calculated gamma correction parameter is less than 1, and the image will be brightened overall after gamma transformation; when the first enhanced image is generally bright (average grayscale value greater than 0.5), the calculated gamma correction parameter is greater than 1, and the image will be darkened overall after gamma transformation. After determining the gamma correction parameter, this embodiment performs gamma transformation on the grayscale value of each pixel in the first enhanced image. Gamma transform is a non-linear brightness adjustment method, mathematically defined as the output equal to the input raised to the power of its gamma. This non-linear transformation can adjust the brightness distribution and contrast characteristics of an image without altering its basic structure: when the gamma value is less than 1, the brightness increase in dark areas is greater than in bright areas, resulting in an overall brighter image with clearer details in the dark areas; when the gamma value is greater than 1, the brightness reduction in bright areas is greater than in dark areas, resulting in an overall darker image with clearer details in the bright areas. The resulting enhanced image, after gamma transform, has a more balanced brightness distribution and a more comfortable overall visual experience.

[0114] S704. Sort the pixel values ​​of the second enhanced image, and remove the pixel values ​​that are above the first preset threshold and below the second preset threshold after sorting to obtain the final corrected image; the first preset threshold is greater than the second preset threshold.

[0115] It should be noted that after the gamma transform, the brightness distribution of the second enhanced image has been optimized, but there may still be some abnormal extreme pixels, i.e., isolated points that are too bright or too dark. These outliers are usually caused by noise, correction residue, or numerical calculation errors during the gamma transform process. If left untreated, they will affect the visual effect of the image and the accuracy of subsequent applications. Therefore, the second enhanced image is sorted by pixel value and outlier removal is performed. Specifically, in this embodiment, the gray values ​​of all pixels in the second enhanced image are arranged in ascending order, and then two preset thresholds are set: a higher first preset threshold is used to remove overly bright outliers, and a lower second preset threshold is used to remove overly dark outliers, wherein the first preset threshold is greater than the second preset threshold.

[0116] In one specific implementation of this embodiment, the first preset threshold can be set as the pixel value located in the top 5% after sorting, and the second preset threshold can be set as the pixel value located in the bottom 5% after sorting. Outliers are removed by: increasing pixels with grayscale values ​​below the second preset threshold (overly dark outliers) to the pixel value corresponding to the second preset threshold; decreasing pixels with grayscale values ​​above the first preset threshold (overly bright outliers) to the pixel value corresponding to the first preset threshold; and leaving normal pixel values ​​between the two thresholds unchanged. Through this dynamic cropping process, this embodiment can effectively remove extreme outliers in the second enhanced image while maintaining the overall brightness distribution characteristics of the image. After all the above steps, this embodiment finally obtains a high-quality final corrected image.

[0117] In summary, gradient-weighted image enhancement is performed on the intermediate corrected image I_correct, followed by adaptive gamma optimization and cropping. Image enhancement can be represented as a regularization problem, and the proposed formula is: ; in, The target image is the enhanced image (to be solved). The original image, i.e., the intermediate corrected image I output after iterative correction. correct , The target gradients (auxiliary variables) are in the horizontal and vertical directions. For target image The actual gradients in the horizontal and vertical directions, The gradient weighting function at position The weight value at that point, This is a regularization parameter that controls the strength of the sparse regularization term. The penalty parameter controls the strength of the constraint terms. These are Lagrange multipliers used in the alternating direction multiplier method for solving problems. It is the L2 norm (Euclidean norm). It is the L1 norm (sum of absolute values), used to achieve gradient sparsity. The gradient weighting function aims to allocate more weights to regions with larger gradients to enhance details, and less weights to regions with smaller gradients to avoid artifacts. Its formula is: (Equation 16); where, The gradient weighting function at position The weight value at that point, This is a constant that controls the strength of the gradient magnitude's contribution to the weights. For the image at position The gradient magnitude at point [0, 1]. This problem can be reduced to a convex optimization problem, commonly solved using the alternating direction multiplier method. This involves iteratively updating B, C, D and T1, T2 respectively, gradually approaching the optimal solution. During the iteration process, the penalty parameter is updated to accelerate convergence. Specifically, in the solution process... The subproblem of B is: (Equation 17); The subproblem of C is: (Equation 18); The subproblems of D are: (Equation 19); Lagrange multipliers T1, T2: , ; in, This represents the deviation between the actual gradient and the target gradient in the horizontal direction. This represents the deviation between the actual gradient and the target gradient in the vertical direction. In this example, only one image enhancement was performed. Furthermore, to further enhance the visual effect, adaptive gamma optimization and cropping were applied to the enhanced image. The formula for adaptive gamma optimization is: ;in, For gamma correction parameters, This is the global average of all pixel values ​​in the image (first average each column, then average each row). The final corrected image output is... ; in, For the second enhanced image at location Pixel value at that location, For the first enhanced image at location The pixel values ​​at each location. After adaptive gamma optimization of the image, outliers may appear. It's necessary to set upper and lower limits for the outlier threshold to exclude high and low pixel values, thus cropping the pixels. In this example, the pixel values ​​for the high and low thresholds are not fixed. Instead, the first five percent and last five percent of the sorted pixels are used as the high and low thresholds. Pixels greater than the high threshold are reduced to the high threshold value, and pixels less than the low threshold are increased to the low threshold value.

[0118] In this embodiment, a complete post-enhancement processing workflow is constructed by combining gradient-weighted sparse regularization with adaptive gamma correction and pixel cropping, which can effectively improve the visual effect and image quality of the corrected image. Furthermore, by constructing a gradient-weighted sparse regularization term and dynamically allocating regularization weights according to the gradient magnitude, the enhancement processing can strengthen sparse constraints in edge and texture regions, thereby highlighting details and sharpening edges, while weakening constraints in flat regions to avoid noise amplification or artifact generation, achieving an adaptive balance between detail enhancement and noise suppression. Furthermore, by employing the alternating direction multiplier method to solve the objective function, the complex L1 sparse optimization problem can be decomposed into multiple efficiently solvable subproblems, achieving a faster convergence speed while ensuring solution accuracy, meeting the processing efficiency requirements of practical applications. Furthermore, by combining adaptive gamma correction and percentile cropping in post-processing, the overall brightness distribution of the image can be further optimized on the basis of gradient-weighted enhancement, and extreme pixel outliers that may be generated during the enhancement process can be eliminated. This makes the contrast of the final output image more harmonious, the pixel distribution more reasonable, and more in line with the visual characteristics of the human eye and the requirements of subsequent recognition tasks. It can further improve the detail performance, contrast and robustness of the image on the basis of the aforementioned thermal radiation correction. The final output image not only effectively removes thermal radiation interference, but also has clear edges, moderate brightness and no obvious abnormalities, and has good visual effects and application value.

[0119] Figure 8 This embodiment demonstrates the intermediate process of BTB surface fitting, wherein... Figure 8 (a) is the original thermal radiation degradation image. Figure 8 (b) is the image after weighted image filtering. Figure 8 (c) is the image after threshold downsampling. Figure 8 (d) is the 3D surface image generated after BTB surface fitting. First, the image to be corrected, affected by aerodynamic thermal radiation, is obtained, such as... Figure 8 As shown in (a), due to the rapid temperature increase of the optical radome of the high-speed aircraft under aerodynamic heating, strong infrared radiation is generated, resulting in a large area of ​​high brightness in the image, a decrease in overall image contrast, and the target details being obscured by thermal radiation. In this embodiment, this image is treated as a degraded image, and grayscale and double-precision processing is performed on it to convert it into a grayscale image of double-precision floating-point type, providing a unified numerical basis for subsequent filtering, fitting, and correction operations. Weighted image filtering processing is performed on the degraded image to obtain the following result: Figure 8The filtered image shown in (b) is as follows. Specifically, in this embodiment, guided filtering and scale-aware filtering are applied to the degraded image to obtain a first filtered image and a second filtered image. Then, based on the normalized grayscale value of the degraded image and the ratio of the gradient magnitudes of the two filtered images, the weights of the guided filtering and scale-aware filtering are calculated, and their sum is 1. Finally, the two filtered images are multiplied by their respective weights and then added together to obtain a weighted fused filtered image. Through the above weighted fusion method, guided filtering is sensitive to dark areas in the image, while scale-aware filtering is sensitive to bright areas. The two complement each other, which effectively suppresses noise in the filtered image, while preserving the large-area low-frequency components of thermal radiation, providing high-quality input for subsequent downsampling and surface fitting. Threshold downsampling is performed on the filtered image to obtain the image shown in (b). Figure 8 (c) shows the low-resolution downsampled image. Specifically, in this embodiment, the filtered image is first divided into multiple 2×2 sampling blocks, and the average gradient magnitude within each sampling block is calculated. When the average gradient magnitude is less than a preset gradient threshold, it is determined to be a simple region, and the pixel value of the upper left corner of the sampling block is directly taken as the downsampled result; when the average gradient magnitude is greater than or equal to the preset gradient threshold, it is determined to be a complex region, and a normalized weight is calculated based on the gradient magnitude of each pixel in the sampling block, and a weighted average is performed on all pixel values ​​in the sampling block to obtain the downsampled result. Through the above adaptive downsampling strategy, nearest neighbor sampling is used in simple regions to reduce the amount of computation, and weighted average sampling is used in complex regions to retain detailed information. The image size after downsampling is about one-quarter of the original image, which greatly reduces the computational complexity of subsequent surface fitting, while maintaining the main shape of the thermal radiation surface. BTB surface fitting is performed on the downsampled image to generate the image shown below. Figure 8(d) shows the three-dimensional fitted surface. Specifically, in this embodiment, the downsampled image is first divided into multiple 3×3 image blocks, and surface fitting is performed on each image block based on the thin plate spline interpolation algorithm: the coordinates and pixel values ​​of each pixel in each image block are extracted, the weighted pixel value is calculated according to the Gaussian weight and the pixel value of the corresponding pixel, the center point of each image block is used as the weight center, the distance from each control point to the weight center is calculated and the Gaussian weight is determined, the Gaussian weight is multiplied by the pixel value of the corresponding control point as the interpolation constraint, the interpolation function parameters that satisfy the interpolation constraint and the zero sum constraint of the linear term coefficient are solved, and the local fitted surface of each image block is obtained. Then, in this embodiment, the boundaries of adjacent image blocks are stitched and fused using a biquadratic Bézier surface interpolation algorithm: For two adjacent image blocks, the function values ​​of the locally fitted surfaces on the adjacent boundaries of the two blocks are extracted as control points of the Bézier surface. A biquadratic Bézier surface interpolation function is constructed, and the pixel values ​​in the boundary regions of the two blocks are interpolated. The interpolation results replace the pixel values ​​at the original block boundaries. Finally, all image blocks are combined in spatial order to obtain a complete surface. Through the above BTB surface fitting, thin-plate spline interpolation ensures the accuracy and C2 continuity of the surface fitting within each block, while biquadratic Bézier surface interpolation ensures a smooth transition between the boundaries of the blocks. The final generated three-dimensional surface accurately depicts the intensity distribution of thermal radiation interference. Figure 8 It can be seen that by combining weighted guided filtering and scale-aware filtering, image noise can be suppressed while preserving the low-frequency components of thermal radiation. By using gradient threshold-based adaptive downsampling, the main shape of the thermal radiation surface can be maintained while significantly reducing the computational load. By combining thin-plate spline interpolation with biquadratic Bézier surface interpolation (BTB), both local fitting accuracy and global boundary continuity can be taken into account, thus providing an accurate thermal radiation surface model for subsequent iterative correction.

[0120] Figure 9 This embodiment illustrates the evolution of the filtered image corresponding to each iteration during the iterative correction process, wherein... Figure 9 (a) to Figure 9 (d) These are the filtered images obtained after weighted image filtering of the current image to be processed in the 1st to 4th iterations, respectively. An iterative method is used to progressively correct thermal radiation. In the first iteration, the current image to be processed is... Figure 8 The original degraded image shown in (a) is obtained after weighted image filtering. Figure 9 The filtered image is shown in (a). Subsequently, in this embodiment, the corrected image of this iteration is used as the current image to be processed in the next iteration, and the weighted image filtering is continued. Figure 9 (b) shows the result of filtering the first corrected image during the second iteration. Figure 9(c) shows the filtering result during the third iteration. Figure 9 (d) shows the filtering result during the fourth iteration. According to... Figure 9 As can be seen, with the increase of the number of iterations, the residual thermal radiation in the current image to be processed decreases successively, and the image quality gradually improves. Therefore, the large areas of high brightness caused by thermal radiation in the filtered image gradually weaken, and the structural details in the image become clearer step by step. Through the above iterative filtering mechanism, the object of each filtering is the image after the previous round of correction and improvement, so that the quality of the filtering result shows a progressive improvement. In this embodiment, by using the corrected image as the input for the next round of filtering and performing iterative processing, the filtering operation can gradually focus on the residual thermal radiation component, thereby improving the quality of the filtered image step by step and providing increasingly accurate input data for subsequent surface fitting.

[0121] Figure 10 This embodiment illustrates the evolution of the corrected image corresponding to each iteration during the iterative correction process, wherein... Figure 10 (a) to Figure 10 (d) These are the corrected images obtained from the 1st to the 4th iterations. In each iteration, correction is performed based on the current image to be processed, the full-size fitted surface, and the coefficient matrix to generate the current corrected image. Specifically, in this embodiment, the gradient magnitude map of the current image to be processed is first calculated and normalized to obtain a smooth hotspot mask. The difference between the current image to be processed and the full-size fitted surface is calculated to obtain a residual image, which is then normalized and subjected to hyperbolic tangent transformation to obtain a residual weight matrix. The dynamic decay factor is calculated based on the ratio of the current iteration number to the maximum iteration number. Then, a coefficient matrix is ​​generated based on the smooth hotspot mask, the residual weight matrix, and the dynamic decay factor. The full-size fitted surface is multiplied by the coefficient values ​​at the corresponding positions in the coefficient matrix to obtain a weighted fitted surface. Finally, the current image to be processed and the weighted fitted surface are subtracted pixel by pixel to obtain the current corrected image. Figure 10 (a) is the corrected image obtained in the first iteration. At this time, the thermal radiation in the image has been initially removed, but there are still obvious residues. Figure 10 (b) is the corrected image obtained in the second iteration, where thermal radiation is further reduced and image contrast is improved; Figure 10 (c) is the corrected image obtained in the third iteration. The residual thermal radiation is no longer obvious, and the target area is clearer. Figure 10 (d) shows the corrected image obtained in the fourth iteration, achieving optimal image quality with minimal thermal radiation removal. Through the aforementioned coefficient matrix correction mechanism, the coefficient matrix is ​​adaptively adjusted based on three factors: a smoothed thermal area mask (reflecting gradient distribution), a residual weight matrix (reflecting the strength of fitting differences), and a dynamic attenuation factor (reflecting the iteration process). This ensures that the intensity of each correction matches the spatial distribution characteristics of the current residual thermal radiation, achieving progressive and adaptive thermal radiation removal. Figure 10It can be seen that by adaptively adjusting the coefficient matrix, the correction intensity can be increased in areas with strong residual thermal radiation and decreased in areas with weak residual thermal radiation, thus avoiding over-correction or under-correction. Through the iterative mechanism, the optimal effect can be gradually approached after each correction, ultimately effectively eliminating thermal radiation interference in the image and significantly improving the image clarity and contrast.

[0122] Figure 11 This demonstrates the correction effect of this embodiment in the presence of a clear reference image, wherein... Figure 11 (a) is an image of thermal radiation degradation. Figure 11 (b) The corrected image for achieving the best results through iteration. Figure 11 (c) is the actual clear image (reference image). Figure 11 (d) shows the variation curves of SSIM and PSNR during the iteration process. When a clear image without thermal radiation interference exists in the same scene, this clear image is used as the reference image. Figure 11 (a) is a degraded image affected by thermal radiation. Figure 11 (c) shows the corresponding clear reference image. In this embodiment, both the degraded image and the reference image undergo grayscale and double-precision processing to calculate the structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) between them in subsequent iterations, serving as image quality evaluation metrics. The iterative correction process described above is then applied... Figure 11 (a) Processing. After each iteration, calculate the SSIM and PSNR values ​​between the current corrected image and the reference image. When the increment of SSIM and PSNR is less than a preset threshold or decreases, the system automatically terminates the iteration and outputs the intermediate corrected image obtained in the most recent iteration before the termination as the final corrected image, such as... Figure 11 As shown in (b). From Figure 11 (b) It can be seen that the corrected image is consistent with Figure 11 The clear reference image shown in (c) is visually very close, with thermal radiation effectively removed and target details clearly discernible. Figure 11 (d) shows the curves of SSIM and PSNR as a function of the number of iterations. The horizontal axis represents the number of iterations, and the vertical axis represents the SSIM and PSNR values. The curves show that as the number of iterations increases, both SSIM and PSNR gradually rise, indicating that the structural similarity and peak signal-to-noise ratio between the corrected image and the reference image are continuously improving. After a certain number of iterations, the rate of increase in the metrics slows down and reaches a peak. If iterations continue thereafter, the metrics may plateau or decline. This embodiment monitors the increase and decrease of the metrics and automatically terminates the iteration when the increment is less than a threshold or a decrease occurs, thereby avoiding invalid iterations and image quality degradation. Figure 11It can be seen that by using SSIM and PSNR as image quality evaluation indicators with reference, the difference between the corrected image and the reference image without thermal radiation can be objectively and quantitatively evaluated. By automatically determining the optimal iteration stopping point by monitoring the increment of the indicators, overfitting or invalid iteration can be avoided, thereby efficiently obtaining the correction result closest to the real scene under the condition of having a reference image.

[0123] Figure 12 This demonstrates the correction effect of this embodiment in the absence of a clear reference image, wherein... Figure 12 (a) is an image of thermal radiation degradation. Figure 12 (b) The corrected image for achieving the best results through iteration. Figure 12 (c) is the final image after image enhancement. Figure 12 (d) shows the curves of SNR and EOG, two no-reference evaluation metrics, as a function of the number of iterations. When no clear image without thermal radiation interference exists in the same scene, the no-reference image quality evaluation metric is used to guide the termination of the iteration. The iteration correction process is followed as described above. Figure 12 (a) Processing. After each iteration, the signal-to-noise ratio (SNR) and gradient energy (EOG) of the corrected image are calculated as reference-free quality evaluation indicators. SNR reflects the ratio of image signal to noise; a higher value indicates better image quality. EOG reflects the gradient energy of the image, i.e., the sharpness of image edges; a higher value indicates a sharper image. When the increment of SNR and EOG is less than a preset threshold or decreases, the system automatically terminates the iteration and outputs the intermediate corrected image obtained from the most recent iteration before the termination as the intermediate corrected image, such as... Figure 12 As shown in (b). From Figure 12 (b) It can be seen that even without a reference image, this embodiment can still effectively remove thermal radiation from the image, and the image contrast and sharpness are significantly improved. Figure 12 (b) shows the intermediate corrected image after post-enhancement processing, resulting in... Figure 12(c) shows the final corrected image. Specifically, this embodiment first constructs an objective function containing a data fidelity term and a gradient-weighted sparse regularization term. The gradient-weighted sparse regularization term assigns weights based on the image gradient magnitude, allocating more weights to regions with larger gradient magnitudes to enhance details and less weights to regions with smaller gradient magnitudes to avoid artifacts. Then, the objective function is solved using the alternating direction multiplier method to obtain the enhanced image. Next, the average gray value of the enhanced image is calculated, and gamma correction parameters are determined based on this average gray value. A gamma transform is then performed on the enhanced image to adjust the overall brightness distribution. Finally, the pixel values ​​of the gamma-transformed image are sorted. Pixel values ​​below the top 5% are raised to their corresponding percentile values, while pixel values ​​above the bottom 5% are lowered to their corresponding percentile values, completing pixel cropping and outputting the final corrected image. Through the above post-enhancement processing, the image contrast is further improved, edges are sharper, and potentially abnormal pixel values ​​are removed, resulting in a significant improvement in visual effect. Figure 12 In (d), the horizontal axis represents the number of iterations, and the vertical axis represents the SNR and EOG values. The curves show that as the number of iterations increases, the SNR and EOG values ​​gradually rise, indicating that the signal-to-noise ratio and edge sharpness of the corrected image continuously improve. After a certain number of iterations, the rate of increase in the indicators slows down and reaches a peak. If iterations continue thereafter, the indicators may plateau or decline. This embodiment monitors the increase and decrease of the indicators and automatically terminates the iteration when the increase is less than a threshold or a decrease occurs, thus achieving the best correction effect even without a reference image. Figure 12 It can be seen that by using SNR and EOG as image quality evaluation indicators in the absence of a reference, the degree of quality improvement of the corrected image can be objectively evaluated. The gradient-weighted sparse regularization image enhancement method can effectively suppress artifacts while enhancing image details. Through adaptive gamma correction and dynamic pixel cropping, the brightness distribution of the image can be further optimized and abnormal pixel values ​​can be removed, so that a high-quality, high-contrast, and sharp-edge final corrected image can still be obtained without a reference image.

[0124] This invention employs a BTB two-stage surface fitting mechanism, which balances the local fitting accuracy of thin-plate spline interpolation with the global boundary continuity of Bézier surface interpolation, thereby accurately characterizing the spatial distribution of thermal radiation interference. Through adaptive adjustment of the coefficient matrix, the correction intensity can be dynamically adjusted based on the gradient distribution of residual thermal radiation, the intensity of fitting differences, and the iteration process, achieving progressive and spatially adaptive thermal radiation removal. An early stopping mechanism automatically determines the optimal stopping point based on SSIM / PSNR when a reference image is available, and on SNR / EOG when no reference image is available, avoiding invalid iterations and image quality degradation. Post-enhancement processing further improves the contrast and edge sharpness of the corrected image while suppressing artifacts and abnormal pixel values.

[0125] To better implement the BTB-based image correction method in the embodiments of the present invention, based on the BTB-based image correction method, correspondingly, as follows: Figure 13 As shown, this embodiment of the invention also provides a BTB-based image correction system 1300, which includes: The preprocessing module 1301 is used to acquire the original image affected by thermal radiation interference and convert the original image into a grayscale image; The weighted filtering module 1302 is used to perform weighted image filtering on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image; The downsampling module 1303 is used to downsample the filtered image to obtain a downsampled image; The fitting module 1304 is used to perform BTB surface fitting on the downsampled image to generate a fitted surface, and restore the fitted surface to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The correction module 1305 is used to generate the current corrected image based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, fitting difference intensity, and current iteration number of the current image to be processed. The iterative processing module 1306 is used to take the current corrected image as the current image to be processed in the next iteration, repeat the above filtering, downsampling, surface fitting and correction steps, calculate the quality evaluation index of the current corrected image after each iteration, and terminate the iteration when the increment of the quality evaluation index is less than a preset threshold or decreases; and perform post-enhancement processing on the intermediate corrected image obtained in the most recent iteration before the termination of the iteration to obtain the final corrected image.

[0126] The BTB-based image correction system 1300 provided in the above embodiments can implement the technical solutions described in the above BTB-based image correction method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above BTB-based image correction method embodiments, and will not be repeated here.

[0127] like Figure 14 As shown, the present invention also provides an electronic device 1400. The electronic device 1400 includes a processor 1401, a memory 1402, and a display 1403. Figure 14 Only some components of the electronic device 1400 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0128] In some embodiments, processor 1401 may be a central processing unit (CPU), microprocessor, or other data processing chip for running program code stored in memory 1402 or processing data, such as the BTB-based image correction method of the present invention.

[0129] In some embodiments, processor 1401 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 1401 may be local or remote. In some embodiments, processor 1401 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, multi-cloud, or any combination thereof.

[0130] In some embodiments, memory 1402 may be an internal storage unit of electronic device 1400, such as a hard disk or memory of electronic device 1400. In other embodiments, memory 1402 may also be an external storage device of electronic device 1400, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on electronic device 1400.

[0131] Furthermore, the memory 1402 may include both internal storage units of the electronic device 1400 and external storage devices. The memory 1402 is used to store application software and various types of data installed on the electronic device 1400.

[0132] In some embodiments, display 1403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 1403 is used to display information from electronic device 1400 and to display a visual user interface. Components 1401-1403 of electronic device 1400 communicate with each other via a system bus.

[0133] In one embodiment, when processor 1401 executes a BTB-based image correction program in memory 1402, the following steps may be performed: Acquire the original image affected by thermal radiation interference, and convert the original image into a grayscale image; A weighted image filtering process is performed on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image. The filtered image is downsampled to obtain a downsampled image; The BTB surface fitting is performed on the downsampled image to generate a fitted surface, and the fitted surface is restored to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The current corrected image is generated based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, the fitting difference intensity, and the current iteration number of the current image to be processed. The corrected image is used as the current image to be processed in the next iteration. The above filtering, downsampling, surface fitting and correction steps are repeated. After each iteration, the quality evaluation index of the corrected image is calculated. The iteration is terminated when the increment of the quality evaluation index is less than the preset threshold or when it decreases. The intermediate corrected image obtained from the most recent iteration before the iteration ends is subjected to post-enhancement processing to obtain the final corrected image.

[0134] It should be understood that when the processor 1401 executes the BTB-based image correction program in the memory 1402, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0135] Furthermore, the embodiments of the present invention do not specifically limit the type of the electronic device 1400 mentioned. The electronic device 1400 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 1400 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0136] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the BTB-based image correction methods provided in the above-described method embodiments.

[0137] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0138] The above provides a detailed description of the image correction method, system, device, and medium based on BTB provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An image correction method based on BTB, characterized in that, include: Acquire the original image affected by thermal radiation interference, and convert the original image into a grayscale image; A weighted image filtering process is performed on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image. The filtered image is downsampled to obtain a downsampled image; The BTB surface fitting is performed on the downsampled image to generate a fitted surface, and the fitted surface is restored to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The current corrected image is generated based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, the fitting difference intensity, and the current iteration number of the current image to be processed. The corrected image is used as the current image to be processed in the next iteration. The above filtering, downsampling, surface fitting and correction steps are repeated. After each iteration, the quality evaluation index of the corrected image is calculated. The iteration is terminated when the increment of the quality evaluation index is less than the preset threshold or when it decreases. The intermediate corrected image obtained from the most recent iteration before the iteration ends is subjected to post-enhancement processing to obtain the final corrected image.

2. The method according to claim 1, characterized in that, The step of performing weighted image filtering on the current image to be processed to obtain the filtered image includes: The current image to be processed is subjected to weighted guided filtering and scale-aware filtering respectively to obtain a first filtered image and a second filtered image; The current image to be processed is normalized to obtain a normalized grayscale image; The first weight and the second weight are calculated based on the gradient magnitudes of the first filtered image and the second filtered image, and the normalized grayscale image. The sum of the first weight and the second weight is 1. The filtered image is obtained based on the first filtered image, the first weight, the second filtered image, and the second weight.

3. The method according to claim 1, characterized in that, The step of downsampling the filtered image to obtain a downsampled image includes: The filtered image is divided into multiple sampling blocks, and the average gradient magnitude within each sampling block is calculated. If the average gradient magnitude of the current sampling block is less than a preset gradient threshold, the pixel value at a specified position in the current sampling block is determined as the downsampling result of the current sampling block. If the average gradient magnitude of the current sampling block is greater than or equal to a preset gradient threshold, a normalized weight is calculated based on the gradient magnitude of each pixel in the current sampling block, and a weighted average is performed on all pixel values ​​in the current sampling block based on the normalized weight to obtain the downsampling result of the current sampling block. Based on the downsampling results of all the sampling blocks, the downsampled image is obtained by combining them according to the spatial arrangement order of each sampling block in the filtered image.

4. The method according to claim 1, characterized in that, The method of performing surface fitting on each image block of the downsampled image based on the thin-plate spline interpolation algorithm includes: The downsampled image is divided into multiple image blocks, and the Gaussian weights of the corresponding pixels in the image block are calculated based on the standard deviation of the Gaussian kernel, the center point of the Gaussian weight function, and the pixels in the image block. The weighted pixel value is calculated based on the Gaussian weight and the pixel value of the corresponding pixel. A thin-plate spline interpolation function is constructed based on the Gaussian weights, the radial basis function of the thin-plate spline interpolation algorithm, and the coordinates of the control points and query points in the image blocks; the control points and the query points are different pixels in the image blocks. Construct interpolation constraints such that the calculated value of the thin plate spline interpolation function at each control point is equal to the weighted pixel value of the control point; Construct zero-sum constraints such that the algebraic sum of the weights of all radial basis functions in the thin plate spline interpolation function is zero, and the algebraic sum of the products of the weights of all radial basis functions and their coordinates is zero; Based on the thin plate spline interpolation function, the interpolation results satisfying the interpolation constraints and the zero-sum constraints within the image block are solved, and the local fitted surface of the image block is generated according to the interpolation results.

5. The method according to claim 4, characterized in that, The step of stitching and fusing the boundaries of adjacent image blocks using a biquadratic Bézier surface interpolation algorithm to obtain the fitted surface includes: For two adjacent image blocks, the function values ​​of the local fitted surfaces of the two image blocks on the adjacent boundaries are extracted and used as Bezier boundary control values. Based on the Bézier boundary control value, a biquadratic Bézier surface interpolation function is constructed to calculate the Bézier fitting value at each pixel position within the boundary region of two adjacent image blocks. The pixel values ​​of two adjacent image blocks within the boundary region are updated based on the Bezier fitting value, and all image blocks are combined in spatial order to obtain a fitted surface corresponding to the downsampled image size.

6. The method according to claim 1, characterized in that, The step of generating the corrected image based on the current image to be processed, the target surface, and the coefficient matrix includes: Calculate the gradient magnitude at each pixel position in the current image to be processed, and normalize the gradient magnitude based on the maximum value of the gradient magnitude to obtain a smooth hotspot mask; The difference between the current image to be processed and the target surface is calculated to obtain a residual image. The residual image is normalized and then transformed to obtain a residual weight matrix. The dynamic decay factor is calculated based on the ratio of the current iteration number to the maximum iteration number. The coefficient matrix is ​​calculated based on the smoothed hot zone mask, the residual weight matrix, and the dynamic attenuation factor. Multiply the target surface by the coefficient value at the corresponding position in the coefficient matrix to obtain the weighted target surface; The current image to be processed is subtracted pixel by pixel from the weighted target surface to obtain the current corrected image.

7. The method according to claim 1, characterized in that, Also includes: Using the intermediate corrected image as the input image, an objective function is constructed. The objective function is used to constrain the enhanced image to be close to the intermediate corrected image, while applying sparse constraints to the gradient of the enhanced image. The regularization weights of the sparse constraints are positively correlated with the gradient magnitude. The objective function is solved using an optimization algorithm to obtain the first enhanced image; Calculate the average gray value of the first enhanced image, determine the gamma correction parameters based on the average gray value, and perform gamma transformation on the first enhanced image based on the gamma correction parameters to obtain the second enhanced image; The second enhanced image is sorted by pixel value, and pixel values ​​that are above a first preset threshold and below a second preset threshold are removed to obtain the final corrected image; the first preset threshold is greater than the second preset threshold.

8. An image correction system based on BTB, characterized in that, include: A preprocessing module is used to acquire the original image affected by thermal radiation interference and convert the original image into a grayscale image; The weighted filtering module is used to perform weighted image filtering on the current image to be processed to obtain a filtered image; in the first iteration, the current image to be processed is the grayscale image; The downsampling module is used to downsample the filtered image to obtain a downsampled image; The fitting module is used to perform BTB surface fitting on the downsampled image to generate a fitted surface, and restore the fitted surface to the original size of the grayscale image to obtain the target surface; wherein, the BTB surface fitting includes performing surface fitting on each image block of the downsampled image based on the thin plate spline interpolation algorithm, and stitching and fusing the boundaries of adjacent image blocks through the biquadratic Bézier surface interpolation algorithm to obtain the fitted surface; The correction module is used to generate the corrected image based on the current image to be processed, the target surface, and the coefficient matrix; the coefficient matrix is ​​generated based on the residual thermal radiation distribution, fitting difference intensity, and current iteration number of the current image to be processed. The iterative processing module is used to take the current corrected image as the current image to be processed in the next iteration, repeat the above filtering, downsampling, surface fitting and correction steps, calculate the quality evaluation index of the current corrected image after each iteration, and terminate the iteration when the increment of the quality evaluation index is less than a preset threshold or decreases; perform post-enhancement processing on the intermediate corrected image obtained in the most recent iteration before the termination of the iteration to obtain the final corrected image.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps of the BTB-based image correction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the BTB-based image correction method according to any one of claims 1 to 7.

Images