An edge-preserving based infrared image digital detail enhancement method and system

Abstract

The application discloses an infrared image digital detail enhancement method and system based on edge preservation. The method first performs non-uniform correction and blind pixel replacement on an original image to generate a reference image; then, a pixel-level three-state grid image is constructed by using multi-scale gradient analysis and noise modeling, adaptive sharpening, bilateral smoothing and hybrid weighting processing are respectively performed on edge, noise and transition state pixels to obtain a state-specific enhancement image. Next, a sparse quadtree is constructed to extract and superimpose a fine detail compensation amount, and then a gray dynamic range adaptive stretching is performed, and finally a global enhancement image is output in real time through fixed-point parallel pipeline. The application successfully resolves the contradiction between edge sharpening and noise smoothing, effectively suppresses noise while preserving fine edges, greatly improves the detail performance and contrast of the infrared image, and meets the real-time imaging requirements.

Description

Technical Field

[0001] The field of infrared image processing technology particularly relates to a method and system for enhancing digital details of infrared images based on edge preservation. Background Technology

[0002] In the field of uncooled infrared thermal imaging technology, image enhancement techniques are widely used in the digital processing of infrared images by uncooled infrared sensors. This technology improves the edge sharpness and detail of infrared images through spatial domain sharpening, multi-scale decomposition, and frequency domain enhancement, while suppressing noise introduced by the detector, thus meeting the image quality requirements of scenarios such as surveillance, security, and search and rescue.

[0003] In practical engineering applications, during the debugging of the AK612 uncooled infrared sensor, it was found that when image detail enhancement was performed to improve target edge sharpness, random noise and water ripple noise in the image were also amplified, especially in uniform background areas. The root cause of this phenomenon is that traditional image enhancement algorithms fail to effectively distinguish between edge details and noise components in an image. They employ a single enhancement or smoothing strategy, resulting in an inability to selectively suppress noise while enhancing edge details, or excessive blurring of edges while suppressing noise. This directly reduces the detection probability and recognition accuracy of targets in the image, potentially leading to missed detections of small or low-contrast targets in complex scenarios, impacting the early warning capabilities of monitoring systems and the efficiency of search and rescue missions. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides the following technical solution:

[0005] An edge-preserving method for digital detail enhancement of infrared images includes:

[0006] Step S1: Perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain a reference infrared image. Perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image to construct a pixel-level three-state labeled raster image.

[0007] Step S2: Based on the pixel-level three-state labeled raster image, perform state-differential enhancement processing on the reference infrared image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-limited bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a state-enhanced image. Perform pixel-by-pixel gradient residual calculation on the state-enhanced image and the reference infrared image to construct a gradient residual compensation sparse quadtree.

[0008] Step S3: Traverse all effective leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it on the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image to generate the global enhanced infrared image.

[0009] Step S4: Perform fixed-point parallel processing pipeline deployment for uncooled infrared cores on the global enhanced infrared image, perform row buffer block scheduling according to the core imaging specifications, realize row-by-row pipelined processing in the field programmable gate array, and output the final enhanced infrared image.

[0010] Further, step S1 includes: S11: acquiring the original infrared image collected by the uncooled infrared detector, performing non-uniform correction on the original infrared image to eliminate detector response non-uniformity, and then performing blind pixel replacement on the non-uniformly corrected image to eliminate bad pixel defects, thereby obtaining a reference infrared image; S12: performing multi-scale gradient calculation on the reference infrared image to obtain a multi-scale gradient amplitude map and a multi-scale gradient direction map; S13: performing local noise statistical modeling on the reference infrared image to obtain a noise variance distribution map; S14: based on the multi-scale gradient amplitude map, the multi-scale gradient direction map, and the noise variance distribution map, performing tri-state classification annotation on each pixel in the reference infrared image to construct a pixel-level tri-state annotation raster map.

[0011] Further, step S12 includes: S121: setting gradient calculation windows at three scales, namely 3x3, 5x5, and 7x7 windows, calculating gray-level differences for each pixel in the reference infrared image along the horizontal and vertical directions under each scale window, to obtain the horizontal gradient component and vertical gradient component at that scale; S122: calculating the gradient magnitude and gradient direction angle for each pixel at each scale based on the horizontal and vertical gradient components, to obtain single-scale gradient magnitude maps and single-scale gradient direction maps at the three scales; S123: performing weighted fusion of the single-scale gradient magnitude maps at the three scales according to scale weights, wherein the scale weights are allocated according to the principle that the weight of the small-scale window is greater than the weight of the large-scale window, and performing weighted fusion of the single-scale gradient direction maps at the three scales according to the same scale weights, to obtain a multi-scale gradient direction map.

[0012] Further, step S13 includes: S131: dividing the reference infrared image into non-overlapping fixed-size statistical blocks, each statistical block being 8 by 8 pixels; S132: calculating the mean and variance of the grayscale values ​​of 64 pixels within each statistical block, and assigning the grayscale variance to all 64 pixels within the statistical block as a local noise variance estimate; S133: performing bilinear interpolation on the local noise variance estimates of all statistical blocks to generate a noise variance distribution map with the same resolution as the reference infrared image.

[0013] Further, step S14 includes: S141: performing improved non-maximum suppression on the multi-scale gradient magnitude map to obtain a suppressed gradient magnitude map; S142: calculating an adaptive double threshold based on the noise variance distribution map; S143: performing tri-state classification labeling on each pixel in the reference infrared image based on the suppressed gradient magnitude map and the adaptive double threshold; S144: performing connectivity checks on all pixels labeled as transitional states, and writing the final labeling results of all pixels into a raster structure with the same resolution as the reference infrared image to form a pixel-level tri-state labeled raster map.

[0014] Further, step S2 includes: S21: Traversing the pixel-level three-state labeled raster image to extract the set of edge-state pixel locations, the set of noise-state pixel locations, and the set of transition-state pixel locations; S22: Performing gradient adaptive sharpening processing on each pixel in the set of edge-state pixel locations; S23: Performing gray-scale domain-restricted bilateral smoothing processing on each pixel in the set of noise-state pixel locations; S24: Performing sharpening and smoothing hybrid weighted processing on each pixel in the set of transition-state pixel locations; S25: Writing the edge sharpening gray value of the edge-state pixel, the noise smoothing gray value of the noise-state pixel, and the transition hybrid gray value of the transition-state pixel back into the same image according to their respective pixel location coordinates to obtain the state-enhanced image; S26: Constructing a gradient residual compensation sparse quadtree based on the state-enhanced image and the reference infrared image.

[0015] Further, step S22 includes: S221: For the current pixel in the edge state pixel location set, read the multi-scale gradient magnitude stored in the corresponding grid cell of the pixel in the pixel-level three-state labeled raster image, traverse the multi-scale gradient magnitude corresponding to all pixels in the edge state pixel location set, take the maximum value among them as the maximum edge gradient magnitude, divide the edge gradient magnitude of the pixel by the maximum edge gradient magnitude to obtain the normalized edge intensity value; S222: Calculate the sharpening gain coefficient of the pixel based on the normalized edge intensity value; S223: Extract the 3x3 neighborhood of the edge state pixel in the reference infrared image, calculate the Laplacian operator response value, multiply the Laplacian operator response value by the sharpening gain coefficient and superimpose it on the original gray value of the pixel in the reference infrared image to obtain the edge sharpening gray value of the pixel.

[0016] Further, step S23 includes: S231: Extracting a 5x5 neighborhood for the noisy pixel in the reference infrared image, calculating the spatial distance between each pixel in the neighborhood and the center pixel, and calculating the spatial domain weight based on the spatial distance and the preset spatial domain standard deviation; S232: Calculating the absolute value of the difference between the gray value of each pixel in the neighborhood and the gray value of the center pixel, reading the noise variance estimate at the position of the center pixel in the noise variance distribution map, taking the square root of the noise variance estimate to obtain the gray domain standard deviation, and calculating the gray domain weight based on the absolute value of the gray value difference and the gray domain standard deviation; S233: Multiplying the gray value of each pixel in the neighborhood by the product of the spatial domain weight and the gray domain weight, summing the weighted gray values ​​of all pixels, and then dividing by the sum of the products of the spatial domain weight and the gray domain weight of all pixels to obtain the noise-smoothed gray value of the pixel.

[0017] Further, step S3 includes: S31: Traversing all effective leaf nodes in the gradient residual compensation sparse quadtree, and reading the region location coordinates, region size, and average pixel gradient residual value stored in each effective leaf node; S32: For each effective leaf node, extracting the corresponding region's pixel block in the reference infrared image according to the region location coordinates and region size, and performing high-pass filtering on the pixel block to extract micro-detail texture information; S33: Superimposing the detail compensation blocks of the corresponding regions of all effective leaf nodes onto the corresponding pixel positions of the fractal enhancement image according to their respective region location coordinates to obtain a detail compensation fused image; S34: Performing grayscale dynamic range adaptive stretching on the detail compensation fused image to obtain a global enhanced infrared image.

[0018] An edge-preserving infrared image digital detail enhancement system is provided to implement the aforementioned edge-preserving infrared image digital detail enhancement method. The system includes:

[0019] The reference image preprocessing and tri-state annotation module is used to perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain the reference infrared image, perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image, and construct a pixel-level tri-state annotation raster image.

[0020] The morphological differentiation enhancement and gradient residual analysis module is used to perform morphological differentiation enhancement processing on the reference infrared image based on the pixel-level three-state labeled raster image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-limited bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a morphological enhancement image. The morphological enhancement image and the reference infrared image are then compared pixel by pixel to calculate the gradient residual and construct a gradient residual compensation sparse quadtree.

[0021] The detail compensation and grayscale stretching module is used to traverse all effective leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it on the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image, and generate a global enhanced infrared image.

[0022] Fixed-point parallel processing deployment module: used to perform fixed-point parallel processing pipeline deployment for uncooled infrared cores on global enhanced infrared images, perform row buffer block scheduling according to the core imaging specifications, realize row-by-row pipelined processing in field programmable gate array and output the final enhanced infrared image.

[0023] This invention constructs a pixel-level three-state labeled raster map and implements a differentiated enhancement strategy for edge, noise, and transition state pixels in uncooled infrared images, fundamentally solving the core technical problem of the contradiction between edge sharpening and noise smoothing in traditional digital detail enhancement algorithms. The method employs weighted fusion of three-scale gradient windows, preserving fine edge information of small targets while suppressing noise interference through a large-scale window. Based on the noise variance distribution map of an 8×8 statistical block, adaptive dual-threshold classification is achieved, and improved non-maximum suppression refines edges to single-pixel width, enabling sub-pixel level edge localization accuracy.

[0024] Gradient adaptive sharpening is applied to edge-state pixels, with strong edges receiving high gain enhancement and weak edges having their sharpening degree controlled by normalized intensity to avoid ringing effects; gray-scale domain-limited bilateral smoothing is performed on noisy-state pixels, with enhanced smoothing intensity in noisy areas and preservation of weak textures in noisy areas; transition-state pixels achieve a smooth and continuous transition through linear mixing weights, eliminating visual discontinuities at processing boundaries.

[0025] Gradient residual compensation using a sparse quadtree accurately locates regions of minor detail loss. High-frequency residuals are extracted through high-pass filtering and weighted according to compensation intensity coefficients, improving the clarity of weak target textures by 30%. Finally, adaptive grayscale dynamic range stretching maps the effective grayscale range of the 14-bit original image to the full dynamic range, enhancing global contrast. This method is implemented on a 100MHz FPGA platform using a 7-line buffered pipeline, improving the detection probability of small targets in surveillance scenarios and the recognition rate of weak-textured targets in search and rescue scenarios. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of an edge-preserving infrared image digital detail enhancement method according to the present invention;

[0028] Figure 2 This is a schematic diagram of multi-scale gradient calculation window fusion in an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram illustrating the generation of the noise variance distribution map in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of a pixel-level three-state labeled raster image in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of grayscale domain-limited bilateral smoothing in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of a gradient residual compensation sparse quadtree in an embodiment of the present invention;

[0033] Figure 7 This is a schematic diagram illustrating the extraction and compensation of fine detail textures in an embodiment of the present invention;

[0034] Figure 8 This is a schematic diagram of a three-stage pipeline parallel processing in an embodiment of the present invention;

[0035] Figure 9 This is a functional block diagram of an edge-preserving infrared image digital detail enhancement system according to the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1:

[0038] Please see Figure 1 As shown, this embodiment provides a method for enhancing digital details in infrared images based on edge preservation, including:

[0039] S1: Perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain the reference infrared image. Perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image to construct a pixel-level three-state labeled raster image.

[0040] Further, step S1 includes:

[0041] S11: Acquire the raw infrared image collected by the uncooled infrared detector, perform non-uniform correction on the raw infrared image to eliminate the non-uniformity of the detector response, and then perform blind pixel replacement on the non-uniformly corrected image to eliminate bad pixel defects, thus obtaining the reference infrared image.

[0042] Further, step S11 includes:

[0043] S111: Raw infrared images are acquired using the uncooled infrared detector built into the AK612 uncooled infrared core. The raw infrared image has a resolution of 640 x 512 and a grayscale depth of 14 bits. The grayscale value of each pixel represents the infrared radiation intensity of the target surface at the corresponding spatial location. The uncooled infrared detector uses a vanadium oxide microbolometer array. Due to the consistency deviation of the microbridge structure manufacturing process and the gain drift of the readout circuit, the electrical signal response of different pixels in the detector array to the same infrared radiation input exhibits non-uniformity. This non-uniformity is directly reflected in the discrete fluctuation of pixel grayscale values ​​within the same temperature region in the raw infrared image. Specifically, in practical application scenarios such as monitoring, security, search and rescue, vehicle-mounted and handheld detection, the target scene temperature difference range faced by the AK612 uncooled infrared core is large. If the non-uniformity of the detector response is not eliminated, the subsequent digital detail enhancement algorithm will be unable to distinguish between the real target edge details and the false grayscale jumps introduced by the non-uniformity, severely reducing the probability of detecting small targets.

[0044] S112: Perform two-point non-uniformity correction on the original infrared image. The specific process of the two-point non-uniformity correction is as follows: During the factory calibration stage, a high-temperature uniform blackbody radiation source and a low-temperature uniform blackbody radiation source are applied to the uncooled infrared detector, respectively, and corresponding high-temperature calibration images and low-temperature calibration images are acquired. For each pixel, the gain correction coefficient and offset correction coefficient are calculated based on the grayscale values ​​in the high-temperature calibration images and low-temperature calibration images. The gain correction coefficient is equal to the array-wide mean of the difference between the high-temperature calibration grayscale value and the low-temperature calibration grayscale value, divided by the difference between the high-temperature calibration grayscale value and the low-temperature calibration grayscale value of the pixel itself. The offset correction coefficient is equal to the array-wide mean of the low-temperature calibration grayscale value minus the product of the gain correction coefficient and the low-temperature calibration grayscale value of the pixel. The grayscale value of each pixel in the original infrared image is multiplied by the gain correction coefficient corresponding to that pixel and then added to the offset correction coefficient to obtain the non-uniformly corrected image.

[0045] S113: Perform blind pixel replacement on the non-uniformly calibrated image. A blind pixel refers to a defective pixel in the uncooled infrared detector array whose response value remains constant or deviates significantly from its neighboring pixels due to manufacturing defects or long-term degradation. The blind pixel detection method is as follows: Read the pixel coordinates of all known blind pixels from the blind pixel location mapping table recorded during the factory calibration stage. Simultaneously, calculate the absolute value of the deviation between each pixel in the non-uniformly calibrated image and the average grayscale value of its 8 neighboring pixels. If this absolute value is greater than a preset blind pixel judgment deviation threshold, the pixel is identified as a dynamic blind pixel and added to the blind pixel location mapping table. The blind pixel judgment deviation threshold is determined based on a preset multiple of the global grayscale standard deviation of the non-uniformly calibrated image. For example, the blind pixel judgment deviation threshold can be set to 3 times the global grayscale standard deviation. For all blind pixel pixels in the blind pixel location mapping table, replace the grayscale value of the blind pixel with the median grayscale value of its 8 neighboring non-blind pixel pixels to obtain the reference infrared image.

[0046] S12: Perform multi-scale gradient calculation on the reference infrared image to obtain a multi-scale gradient magnitude map and a multi-scale gradient direction map.

[0047] Further, step S12 includes:

[0048] S121: Set three gradient calculation windows at three scales: a 3x3 window, a 5x5 window, and a 7x7 window. For each pixel in the reference infrared image within each scale window, calculate the gray-level difference along both the horizontal and vertical directions to obtain the horizontal and vertical gradient components at that scale. Specifically, for the 3x3 window, extract a 3x3 neighborhood centered on the current pixel. The horizontal gradient component equals the sum of the gray values ​​of the three pixels in the right column of the neighborhood minus the sum of the gray values ​​of the three pixels in the left column of the neighborhood. The vertical gradient component equals the sum of the gray values ​​of the three pixels in the downward direction of the neighborhood minus the sum of the gray values ​​of the three pixels in the upward direction of the neighborhood. This calculation method is equivalent to the difference logic of the Prewitt operator. For a 5x5 window, a 5x5 neighborhood is extracted centered on the current pixel. The horizontal gradient component equals the sum of the gray values ​​of the rightmost two columns (10 pixels) minus the sum of the gray values ​​of the leftmost two columns (10 pixels). The vertical gradient component equals the sum of the gray values ​​of the bottom two rows (10 pixels) minus the sum of the gray values ​​of the top two rows (10 pixels). For a 7x7 window, a 7x7 neighborhood is extracted centered on the current pixel. The horizontal gradient component equals the sum of the gray values ​​of the rightmost three columns (21 pixels) minus the sum of the gray values ​​of the leftmost three columns (21 pixels). The vertical gradient component equals the sum of the gray values ​​of the bottom three rows (21 pixels) minus the sum of the gray values ​​of the top three rows (21 pixels). Specifically, three gradient calculation windows of different scales are used because edge details in uncooled infrared images have multi-scale characteristics: small-scale windows are sensitive to fine edges and small target contours, but are also sensitive to noise; large-scale windows are stable in response to macroscopic edges and have strong noise resistance, but will blur the fine edges of small targets. In monitoring and search and rescue scenarios, there are both narrow edges of small distant targets that need to be accurately identified, and wide edges of large targets such as buildings and vehicles. Gradient calculation at a single scale cannot simultaneously cover the detection needs of both types of edges.

[0049] S122: For each pixel at each scale, calculate the gradient magnitude and gradient direction angle based on the horizontal and vertical gradient components, resulting in a single-scale gradient magnitude map and a single-scale gradient direction map at three scales. Specifically, let the horizontal gradient component of a pixel at a certain scale be... The vertical gradient component is Then the gradient magnitude of the pixel at that scale The calculation formula is:

[0050]

[0051] The gradient direction angle of this pixel at this scale The calculation formula is:

[0052]

[0053] The gradient magnitude The gradient reflects the degree of grayscale change at that pixel; a larger gradient magnitude indicates a more pronounced grayscale jump at that pixel, making it more likely to be at the edge of a target; the gradient direction angle... The direction reflecting the most significant grayscale change is vertically to the edge direction. The above calculations are performed on all pixels in the reference infrared image using 3x3, 5x5, and 7x7 windows respectively, resulting in three single-scale gradient magnitude maps and three single-scale gradient direction maps. For example, assuming a pixel in the reference infrared image has a horizontal gradient component of 120 and a vertical gradient component of 90 in a 3x3 window, then the gradient magnitude of that pixel in the 3x3 window is... The gradient direction angle is Spend.

[0054] S123: The single-scale gradient magnitude maps at three scales are weighted and fused according to scale weights, whereby the scale weights are allocated based on the principle that the weight of a smaller scale window is greater than the weight of a larger scale window. Let the scale weight of a 3x3 window be... The scale weight of the 5x5 window is The scale weight of a 7x7 window is ,satisfy and The scale weights are determined as follows: Small target detection is a core requirement in uncooled infrared images; small-scale windows have the highest sensitivity for detecting the edges of small targets, therefore they are given the largest weight to preserve the fine edge information of small targets. Large-scale windows provide stable gradient estimation for macroscopic edges and help suppress noise in the gradient response, therefore they are given smaller weights. For example, It can be set to 0.5. It can be set to 0.3. It can be set to 0.2. The fusion process is as follows: for each pixel in the reference infrared image, multiply the single-scale gradient magnitude of that pixel in a 3x3 window by... Add the single-scale gradient magnitude of that pixel within a 5x5 window multiplied by Add the single-scale gradient magnitude of the pixel within a 7x7 window multiplied by The multi-scale gradient magnitude of the pixel is obtained. The multi-scale gradient magnitudes of all pixels are then combined to form a multi-scale gradient magnitude map with the same resolution as the reference infrared image. The single-scale gradient patterns at the three scales are then weighted equally according to their scale. , , Weighted fusion is performed by multiplying the gradient direction angles of each pixel at each of the three scales by their respective scale weights and then summing the results to obtain the multi-scale gradient direction angles of that pixel. The multi-scale gradient direction angles of all pixels are then combined to form a multi-scale gradient direction map. For example, assuming a pixel has a gradient magnitude of 150 in a 3x3 window, 130 in a 5x5 window, and 110 in a 7x7 window, then the multi-scale gradient magnitude of that pixel is... See also Figure 2 This figure illustrates the multi-scale gradient calculation window fusion provided in this application embodiment. As shown, the figure displays three gradient calculation windows of different scales: 3x3, 5x5, and 7x7, and their corresponding weighted fusion processes. The three windows are arranged from left to right in ascending order of scale. The central black dot of each window represents the position of the pixel to be calculated. Below each window are its respective scale weights: w1 = 0.5, w2 = 0.3, and w3 = 0.2. Arrows at the bottom indicate the direction of increasing window scale. A simple human icon on the left represents the fine edges corresponding to small, distant targets, while a simple vehicle icon on the right represents the macroscopic edges corresponding to large targets, visually illustrating the scene adaptation basis of the multi-scale design. The single-scale gradient magnitudes of the three windows are calculated by the weighted fusion box using the formula w1M1 + w2M2 + w3M3 to output the multi-scale gradient magnitude. In practical applications of uncooled infrared imaging, monitoring and search and rescue scenarios simultaneously present narrow edges for small targets such as distant personnel and wide edges for large targets such as buildings and vehicles. A single-scale window cannot simultaneously meet the detection requirements of both types of edges. This multi-scale fusion scheme retains the sensitive response of small-scale windows to fine edges of small targets with a larger weight, while reducing the interference of noise on gradient estimation by leveraging the noise resistance of large-scale windows, thus achieving an effective balance between edge detection sensitivity and noise resistance stability.

[0055] S13: Perform local noise statistical modeling on the reference infrared image to obtain the noise variance distribution map.

[0056] Further, step S13 includes:

[0057] S131: The reference infrared image is divided into non-overlapping statistical blocks of fixed size, each block being 8 x 8 pixels. The reference infrared image has a resolution of 640 x 512, divided into 80 statistical blocks horizontally and 64 statistical blocks vertically, for a total of 5120 statistical blocks. The 8 x 8 pixel size of the statistical blocks is set as follows: if the statistical blocks are too small, the number of pixels within each block is insufficient for reliable variance estimation; if the statistical blocks are too large, the spatial resolution of the noise variance decreases, failing to reflect local changes in noise intensity. An 8 x 8 pixel statistical block contains 64 sampling points, which is statistically sufficient for effective estimation of local noise variance. Furthermore, 8 is an integer power of 2, which is beneficial for subsequent address calculation via shift operations in a field-programmable gate array (FPGA).

[0058] S132: Calculate the mean and variance of the grayscale values ​​of the 64 pixels within each statistical block. Specifically, let the grayscale values ​​of the 64 pixels within a statistical block be... The grayscale mean of this statistical block The calculation formula is:

[0059]

[0060] The gray variance of this statistical block The calculation formula is:

[0061]

[0062] grayscale variance All 64 pixels within the statistical block are assigned as local noise variance estimates. The physical basis for using grayscale variance as the local noise variance estimate is as follows: In flat areas of uncooled infrared images, fluctuations in pixel grayscale values ​​are mainly caused by random noise from the detector, and grayscale variance directly reflects the energy level of the noise; in areas containing edges, grayscale variance contains both edge transition and noise information, and the grayscale variance here will be higher than the pure noise variance, but this higher variance is naturally diluted by global mean statistics in the adaptive dual threshold calculation in subsequent step S14, and will not affect the accuracy of tri-state classification.

[0063] S133: Perform bilinear interpolation on the local noise variance estimates of all 5120 statistical blocks to generate a noise variance distribution map with the same resolution as the reference infrared image. Specifically, the bilinear interpolation process is as follows: treat the local noise variance estimate of each statistical block as a sampled value at the center of that block. For any pixel in the reference infrared image, determine the center positions of the four nearest statistical blocks. Perform bilinear weighting based on the horizontal and vertical distance ratios between the pixel and the four statistical block center positions to obtain the interpolated noise variance estimate at that pixel. Combine the interpolated noise variance estimates of all pixels to form a noise variance distribution map. The purpose of bilinear interpolation is to eliminate abrupt changes in the noise variance estimates at the boundaries of the statistical blocks, making the noise variance distribution map spatially continuous and smooth, thereby providing a smooth noise baseline for the adaptive dual-threshold calculation in the subsequent step S14. See also... Figure 3 This is a schematic diagram illustrating the generation of a noise variance distribution map provided in this application embodiment. As shown in the figure, this diagram illustrates the complete process from a reference infrared image to the generation of a continuous and smooth noise variance distribution map through local statistical modeling. The left side is a thumbnail representation of the reference infrared image, with non-overlapping 8x8 pixel statistical blocks divided by dashed lines. One of these blocks is highlighted with a thick border. A magnified view extends from this highlighted block, showing the internal structure of the 8x8 pixel statistical block containing 64 pixels. The calculation processing box below indicates the operation of performing grayscale mean and variance calculations on this block to extract local noise variance. After bilinear interpolation, the right side generates a noise variance distribution map with the same resolution as the reference infrared image. This map uses dot matrix of different densities to represent the spatial variation of noise intensity. Dense dot matrix areas correspond to local locations with stronger noise, while sparse dot matrix areas correspond to local locations with weaker noise, presenting an overall continuous and smooth transition effect. In the actual operation of uncooled infrared detectors, due to differences in the fabrication process of the microbridge structure and the drift of the readout circuit, the noise level in different regions of the detector array varies spatially. The noise variance distribution map accurately quantifies this spatial difference, providing a continuous and smooth noise benchmark for the subsequent adaptive double threshold calculation and adaptive selection of the gray-scale standard deviation in gray-scale limited bilateral smoothing, enabling the algorithm to automatically adapt to the actual noise level in different regions.

[0064] S14: Based on the multi-scale gradient magnitude map, multi-scale gradient direction map and noise variance distribution map, perform tri-state classification and labeling on each pixel in the reference infrared image to construct a pixel-level tri-state labeled raster map.

[0065] Further, step S14 includes:

[0066] S141: Perform improved non-maximum suppression on the multi-scale gradient magnitude map to obtain the suppressed gradient magnitude map. Specifically, for each pixel in the multi-scale gradient magnitude map, read the multi-scale gradient direction angle corresponding to that pixel in the multi-scale gradient direction map, and quantize the multi-scale gradient direction angle to one of four discrete directions: 0 degrees for the horizontal direction, 45 degrees for the upper right to lower left direction, 90 degrees for the vertical direction, and 135 degrees for the upper left to lower right direction. The quantization rule is to map the multi-scale gradient direction angle to the closest discrete direction. Along the quantized gradient direction, determine the two adjacent pixels of the pixel in that direction, and compare the multi-scale gradient magnitude of the pixel with the multi-scale gradient magnitude of the two adjacent pixels: if the multi-scale gradient magnitude of the pixel is greater than or equal to the larger of the multi-scale gradient magnitudes of the two adjacent pixels, then retain the multi-scale gradient magnitude of the pixel; otherwise, set the multi-scale gradient magnitude of the pixel to 0. The gradient magnitudes of all pixels after the above operations are used to form the suppressed gradient magnitude map. The physical meaning of the improved nonmaximum suppression is that only the local strongest gradient response is preserved along the gradient direction. This operation refines the gradient response from a wide edge band to a single pixel-width edge line, which is beneficial for subsequent accurate sharpening of edge-state pixels and avoids performing unnecessary sharpening operations on pixels with non-edge peaks.

[0067] S142: Calculate the adaptive dual threshold based on the noise variance distribution map. Specifically, iterate through the noise variance estimates of all pixels in the noise variance distribution map, calculate the arithmetic mean of all noise variance estimates, and denot it as the global noise root mean square error. The global noise mean square error Multiply by the high threshold coefficient Obtain high threshold The global noise mean square error Multiply by the low threshold coefficient Obtain low threshold The calculation formula is:

[0068]

[0069]

[0070] The high threshold coefficient Greater than the low threshold coefficient And all are preset positive numbers. High threshold coefficient The determination is based on: making the high threshold It can effectively distinguish between gradient responses at real edges and gradient responses caused by noise, with a high threshold coefficient. Too small a threshold will cause noisy pixels to be mislabeled as edge states, while too large a threshold will cause weak edge pixels to be missed. Low threshold coefficient The determination is based on: making the low threshold It can distinguish the gradient response of a purely noisy region from a transition region that may contain weak edges. For example, a high threshold coefficient... It can be set to 4.0, a low threshold coefficient. It can be set to 1.5. For example, assume the global noise mean square error... If the value is 25, then the high threshold is... low threshold The core advantage of adaptive dual thresholds lies in the direct correlation between the thresholds and the noise level: when the detector noise is high, the global noise mean square error increases, and the dual thresholds are raised accordingly, avoiding misjudging a large amount of noise as edges; when the detector noise is low, the dual thresholds are lowered, retaining more weak edge information. This adaptive mechanism enables the algorithm to automatically adapt to changes in detector noise levels under different ambient temperatures and different operating times, without the need for manual parameter adjustment.

[0071] S143: Based on the suppressed gradient magnitude map and adaptive dual thresholds, perform tri-state classification annotation on each pixel in the reference infrared image. Specifically, iterate through each pixel in the suppressed gradient magnitude map and read the gradient magnitude of that pixel in the suppressed gradient magnitude map: if the gradient magnitude is greater than or equal to the high threshold... If the gradient magnitude is less than the low threshold, then the pixel is marked as an edge state; If the gradient magnitude is greater than or equal to the low threshold, then the pixel is marked as a noise state; And less than the high threshold If the pixel is in a transitional state, it is labeled as such. The physical meaning of this three-state classification is as follows: edge-state pixels correspond to the real target edge, with a gradient response significantly higher than the noise level; subsequent sharpening enhancement should be performed to improve edge contrast. Noise-state pixels correspond to flat or purely noisy regions, with a gradient response lower than the noise baseline; subsequent smoothing processing should be performed to suppress noise. Transitional-state pixels are located in the ambiguous zone between the edge and noise, and may be either weak edges or strong noise; subsequent sharpening and smoothing hybrid processing is required based on the gradient magnitude's position between the two thresholds. See also... Figure 4This is a schematic diagram of a pixel-level three-state labeled raster image provided in an embodiment of this application. As shown in the figure, this figure uses a 10x10 pixel local raster region as an example to intuitively demonstrate the spatial distribution results of each pixel in the reference infrared image being labeled as one of three states after multi-scale gradient analysis, adaptive dual-threshold comparison, and connectivity test. Each small square in the figure corresponds to a pixel position. The squares marked with short diagonal lines represent edge state pixels, corresponding to pixels with gradient magnitudes greater than or equal to the high threshold at the real target contour; the squares marked with small dots represent noise state pixels, corresponding to pixels with gradient magnitudes less than the low threshold in flat areas; and the squares marked with thin crosses represent transition state pixels, whose gradient magnitudes are between the low and high thresholds, located in the blurred zone between edges and noise. The dashed lines drawn along the edge state pixels demonstrate the effect of the connectivity test, that is, adjacent edge state pixels form continuous edge trajectories in space. In uncooled infrared imaging applications such as surveillance, security, and search and rescue, this tri-state labeled raster map distinguishes target edges, background noise, and blurred transition regions with pixel-level precision. This allows subsequent state-specific differentiation enhancement processing to apply corresponding sharpening or smoothing strategies to different types of pixels, fundamentally avoiding the contradiction between edge sharpening and noise suppression in traditional single-processing strategies. The legend on the right side of the figure also shows the relationship between gradient magnitude and high and low thresholds, clearly demonstrating the threshold logic of tri-state classification.

[0072] S144: Perform a connectivity check on all pixels labeled as transitional states. Specifically, iterate through all pixels labeled as transitional states and check if there is at least one pixel labeled as edge state within each transitional state pixel's 8-neighborhood. If so, relabel the transitional state pixel as edge state; otherwise, keep the transitional state label unchanged. The physical basis of the connectivity check is that the true target edge is spatially continuous. If a transitional state pixel is adjacent to a confirmed edge state pixel, the transitional state pixel is likely to belong to the continuation of the same edge, rather than an isolated noise point. Upgrading it to edge state can maintain the integrity and continuity of the edge. Write the final labeling results of all pixels into a grid structure with the same resolution as the reference infrared image. The grid structure is a two-dimensional array, where each grid cell corresponds to the position coordinates of a pixel in the reference infrared image. Each grid cell stores two fields: the first field is a three-state labeling value, which can be one of edge state, noise state, or transitional state; the second field is the multi-scale gradient magnitude of the pixel in the multi-scale gradient magnitude map. The completed raster structure is denoted as a pixel-level three-state labeled raster image.

[0073] In step S1, the multi-scale gradient analysis and noise statistical modeling work together to construct a pixel-level tri-state labeled raster image, forming a complete edge-noise differentiation system from the signal level to the decision level. Specifically, the multi-scale gradient calculation in step S12 provides each pixel with a multi-scale gradient magnitude reflecting the drastic change in grayscale and a multi-scale gradient direction angle reflecting the direction of change. The noise statistical modeling in step S13 provides a quantitative benchmark for the global noise level. Step S14 combines the gradient information with the noise benchmark to generate an adaptive dual threshold and complete the tri-state classification. Without the multi-scale gradient calculation in step S12, step S14 would not be able to obtain the gradient response intensity information of each pixel, making it impossible to determine whether the pixel belongs to an edge or noise. Without the noise statistical modeling in step S13, the dual thresholds in step S14 would not be able to adaptively adjust according to the actual noise level of the current image, and the classification accuracy in different noise environments would be severely reduced. In step S12, gradient windows of three scales are used and weighted fusion is performed according to the principle that the weight of small scales is greater than that of large scales. This design addresses the characteristic of multi-scale coexistence of edge details in uncooled infrared images. While preserving the fine edge information of small targets, it leverages the noise resistance of large-scale windows to reduce the interference of noise on gradient estimation. Step S13 uses an 8x8 statistical block to estimate the local noise variance and generates a continuous noise variance distribution map through bilinear interpolation. This method balances the statistical reliability and spatial resolution of noise estimation, enabling the subsequent adaptive dual threshold to accurately reflect the global noise level. Step S14 refines the gradient response to single-pixel-width edge lines through improved non-maximum suppression, and then completes tri-state classification through adaptive dual threshold and connectivity test. The classification result is output in the form of a pixel-level tri-state labeled raster map, providing a precise pixel-level processing strategy for the state differentiation enhancement processing in step S2. The pixel-level three-state labeled raster map accurately distinguishes three types of pixels in space: edge state, noise state, and transition state. It fundamentally solves the core technical problem of the contradiction between edge sharpening and noise smoothing caused by the use of a single processing strategy for all pixels in traditional digital detail enhancement algorithms, and enables subsequent steps to apply differentiated enhancement strategies to different types of pixels.

[0074] S2: Based on the pixel-level three-state labeled raster image, perform state-differentiated enhancement processing on the reference infrared image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-restricted bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a state-enhanced image. Perform pixel-by-pixel gradient residual calculation on the state-enhanced image and the reference infrared image to construct a gradient residual compensation sparse quadtree.

[0075] Further, step S2 includes:

[0076] S21: Traverse the pixel-level tri-state labeled raster image, read the tri-state label value stored in each raster cell, extract the pixel position coordinates of all pixels with tri-state label values ​​of edge state to form an edge state pixel position set, extract the pixel position coordinates of all pixels with tri-state label values ​​of noise state to form a noise state pixel position set, and extract the pixel position coordinates of all pixels with tri-state label values ​​of transition state to form a transition state pixel position set. The edge state pixel position set, the noise state pixel position set, and the transition state pixel position set do not overlap, and their union covers the position coordinates of all pixels in the reference infrared image.

[0077] S22: Perform gradient adaptive sharpening on each pixel in the set of edge state pixel locations.

[0078] Further, step S22 includes:

[0079] S221: For the current pixel in the edge-state pixel location set, read the multi-scale gradient magnitude stored in the corresponding raster cell of the pixel in the pixel-level three-state labeled raster image, and record it as the edge gradient magnitude of the pixel. Iterate through the multi-scale gradient magnitudes corresponding to all pixels in the edge-state pixel location set, and take the maximum value as the maximum edge gradient magnitude. Divide the edge gradient magnitude of the pixel by the maximum edge gradient magnitude to obtain the normalized edge intensity value. The calculation formula is as follows:

[0080]

[0081] in This represents the edge gradient magnitude of the pixel. This represents the maximum magnitude of the edge gradient. Normalized edge strength value. The value of is a closed interval between 0 and 1. The larger the value, the stronger the gradient response of the edge pixel, and the higher the corresponding edge contrast. The physical meaning of normalization is: to map the gradient magnitude of different edge pixels to a uniform scale, so that the subsequent sharpening gain is proportional to the relative intensity of the edge. Strong edges receive a larger sharpening gain to highlight the contour, while weak edges receive a smaller sharpening gain to avoid over-sharpening and introducing ringing effects.

[0082] S222: Based on normalized edge strength values Calculate the sharpening gain coefficient for this pixel. The calculation formula is:

[0083]

[0084] in The preset sharpening base gain, The preset minimum sharpening gain, the sharpening base gain and sharpening minimum gain All are preset positive values. Sharpen base gain. The determination is based on: controlling the dynamic range of sharpening gain as edge intensity changes, and sharpening the base gain. The larger the value, the greater the difference in sharpening gain between strong and weak edges. For example, the sharpening base gain... It can be set to 1.5. Sharpening minimum gain. The determination is based on ensuring that even edge-state pixels with a normalized edge intensity value of 0 can obtain a minimum sharpening enhancement. For example, the minimum sharpening gain. It can be set to 0.5. This sharpens the gain coefficient. The range of values ​​is arrive The closed interval is from 0.5 to 2.0. For example, suppose the normalized edge intensity value of a certain edge-state pixel is... If it is 0.8, then the sharpening gain coefficient is... .

[0085] S223: Extract a 3x3 neighborhood from the reference infrared image for this edge-state pixel and calculate the Laplacian operator response value. The Laplacian operator uses a standard 4-neighborhood Laplacian kernel, and its response value... The calculation formula is:

[0086]

[0087] in This represents the grayscale value of the pixel in the reference infrared image. , , , These are the grayscale values ​​of the four adjacent pixels (top, bottom, left, and right) in the reference infrared image. The Laplacian operator response value... Multiply by sharpening gain factor The original grayscale value of that pixel in the reference infrared image is then superimposed onto it. The edge sharpening grayscale value of the pixel is obtained above. The calculation formula is:

[0088]

[0089] The physical meaning of the Laplacian operator is a second-order differential operator. Its response value reflects the local curvature of the gray value at the pixel. At the edge position, the rate of change of gray value changes abruptly, and the response value of the Laplacian operator deviates significantly from 0. After scaling the response value according to the sharpening gain coefficient, it is superimposed back to the original gray value, which is equivalent to enhancing the gray value jump at the edge and improving the gray value contrast on both sides of the edge.

[0090] S23: Perform grayscale-limited bilateral smoothing on each pixel in the set of noisy pixel locations.

[0091] Further, step S23 includes:

[0092] S231: Extract a 5x5 neighborhood containing 25 pixels from the reference infrared image for this noisy pixel. Calculate the spatial distance between each pixel in the neighborhood and the center pixel. The spatial distance This is the Euclidean distance between the coordinates of neighboring pixels and the coordinates of the center pixel. (Based on spatial distance) and the preset spatial domain standard deviation Calculate spatial domain weights The calculation formula is:

[0093]

[0094] The spatial domain standard deviation The determination is based on: the decay rate of the Gaussian weights in the spatial domain, and the standard deviation in the spatial domain. The larger the value, the higher the weight of distant neighboring pixels and the stronger the smoothness. For example, the spatial domain standard deviation... It can be set to 1.5.

[0095] S232: Calculate the absolute value of the difference between the gray value of each pixel in the neighborhood and the gray value of the center pixel. Read the noise variance estimate at the center pixel location in the noise variance distribution map, and take the square root of the noise variance estimate to obtain the grayscale standard deviation. Based on the absolute value of the difference in grayscale values and grayscale standard deviation Calculate grayscale weights The calculation formula is:

[0096]

[0097] The standard deviation of the grayscale range The value is taken from the noise variance distribution map rather than a fixed preset value. Its physical meaning is: in areas with strong noise, the standard deviation of the gray range is... Larger, grayscale weight It has a higher tolerance for grayscale differences, allowing neighboring pixels with larger grayscale differences to participate in smoothing calculations, thereby enhancing the smoothing effect to suppress stronger noise; in areas with weaker noise, the grayscale standard deviation... Smaller, grayscale weight The tolerance for grayscale differences is reduced, allowing only neighboring pixels with similar grayscale values ​​to participate in smoothing, thus preserving subtle grayscale variations within the region while suppressing weak noise. This mechanism enables the smoothing intensity of grayscale-limited bilateral smoothing to adaptively adjust with the noise level, unlike traditional bilateral filtering where the grayscale standard deviation is a globally fixed value. See also Figure 5 This figure illustrates a schematic diagram of gray-scale domain-restricted bilateral smoothing provided in this application embodiment. As shown, the figure demonstrates the complete computational structure for performing gray-scale domain-restricted bilateral smoothing on noisy pixels. The left side represents a 5x5 neighborhood grid centered on the current noisy pixel. The center pixel is filled in gray and marked with a leader line, while surrounding neighboring pixels are marked with black dots. Two dashed paths extend from the center pixel to the neighboring pixels: one marking spatial distance for calculating spatial domain weights, and the other marking gray-scale difference for calculating gray-scale domain weights. The two weights are calculated in the spatial domain weight calculation box and the gray-scale domain weight calculation box on the right, respectively. The gray-scale domain weight calculation box obtains the gray-scale domain standard deviation from the noise variance distribution map box below via connecting lines, demonstrating the adaptive mechanism of taking the gray-scale domain standard deviation from the noise variance distribution map. After multiplication, the two weights enter the neighborhood weighted summation box, ultimately outputting the noise-smoothed gray-scale value. The label "Provides gray-scale domain standard deviation" in the figure indicates the data flow from the noise variance distribution map to the gray-scale weight calculation parameters. In practical applications of uncooled infrared sensors, noise levels vary in different regions of the detector. This adaptive mechanism enables areas with higher noise levels to automatically receive greater smoothing to fully suppress noise, while areas with lower noise levels automatically reduce smoothing to retain any possible subtle grayscale changes. This differs from the traditional bilateral filtering approach where the grayscale standard deviation is a globally fixed value.

[0098] S233: Multiply the gray value of each pixel in the neighborhood by the spatial domain weight. With grayscale weight The product of the spatial domain weights and grayscale domain weights of all 25 pixels is summed, and then divided by the sum of the products of the spatial domain weights and grayscale domain weights of all 25 pixels to obtain the noise-smoothed grayscale value of that pixel. The calculation formula is:

[0099]

[0100] in This represents a 5x5 neighborhood centered on the pixel in the noisy state. This represents the grayscale value of the neighboring pixels in the reference infrared image.

[0101] S24: Perform sharpening, smoothing, and weighted blending on each pixel in the transition state pixel location set.

[0102] Further, step S24 includes:

[0103] S241: Read the multi-scale gradient magnitude stored in the raster cell corresponding to the transition state pixel in the pixel-level three-state labeled raster image, and base the multi-scale gradient magnitude on the low threshold value. and high threshold Calculate the sharpening blend weights based on the linear interpolation positions between them. The calculation formula is:

[0104]

[0105] in This represents the multi-scale gradient magnitude of the transition state pixel. Since the multi-scale gradient magnitude of the transition state pixel is greater than or equal to the low threshold in the suppressed gradient magnitude map, this is relevant. And less than the high threshold Therefore, sharpen the blend weights. The value range is an open interval from 0 to 1. Sharpening blend weights. The closer the gradient response is to 1, the closer it is to the edge state, and more sharpening should be applied; sharpen the blending weights. The closer the gradient response is to 0, the closer it is to the noise state, and more smoothing should be applied.

[0106] S242: Calculate the edge sharpening grayscale value for the transition state pixel according to the methods in steps S221 to S223. In step S221, the normalized edge strength value The calculation still uses the maximum value of the edge gradient magnitude. As the divisor, calculate the noise-smoothed grayscale value for the transition state pixel according to the methods in steps S231 to S233. .

[0107] S243: Sharpen edge grayscale values Multiply by sharpening blend weights Smooth the grayscale values ​​of the noise. Multiply The sum of the two values ​​yields the transition blending grayscale value of the pixel. The calculation formula is:

[0108]

[0109] The physical meaning of the weighted blending of sharpening and smoothing is as follows: Transitional pixels are located in the ambiguity zone between edges and noise, and cannot be simply classified as pure edges or pure noise. A linear interpolation method is used to blend the sharpening and smoothing results, so that transitional pixels with stronger gradient responses are more inclined to be sharpened, and transitional pixels with weaker gradient responses are more inclined to be smoothed. This achieves a smooth transition from edges to noise and avoids visual discontinuity caused by abrupt changes in processing strategies between edges and flat areas.

[0110] S25: Write the edge sharpening gray values ​​of all pixels in the edge state pixel location set, the noise smoothing gray values ​​of all pixels in the noise state pixel location set, and the transition mixing gray values ​​of all pixels in the transition state pixel location set back into the same 640x512 resolution image according to their respective pixel location coordinates to obtain the state-enhanced image.

[0111] S26: Construct a gradient residual compensated sparse quadtree based on the fractal enhanced image and the reference infrared image.

[0112] Further, step S26 includes:

[0113] S261: Calculate the horizontal and vertical gradient components of each pixel using a 3x3 window for the reference infrared image and the state-enhanced image, respectively, and calculate the gradient magnitude of each pixel according to the gradient magnitude calculation method in step S122, so as to obtain the reference gradient magnitude map and the enhanced gradient magnitude map respectively.

[0114] S262: Subtract the gradient magnitude of the corresponding pixel in the enhanced gradient magnitude map from the gradient magnitude of each pixel in the baseline gradient magnitude map, and take the absolute value to obtain the gradient residual map. The value of each pixel in the gradient residual map reflects the degree of gradient change of that pixel before and after the fractional enhancement process: areas with larger gradient residual values ​​indicate that the fractional enhancement process has significantly changed the gradient structure of that area, which may result in the smoothing and weakening of edge details or the sharpening and amplification of noise, requiring further compensation.

[0115] S263: Construct a gradient residual compensation sparse quadtree based on the gradient residual map. The quadtree is a hierarchical spatial partitioning data structure, where the root node corresponds to the entire image region of the gradient residual map. Specifically, the gradient residual map is used as the entire image region corresponding to the root node, and the region size is 640 x 512. The maximum value of the gradient residual values ​​of all pixels in this region is calculated. If the maximum value is greater than a preset residual significance threshold, the region is divided into two halves along both the horizontal and vertical directions, resulting in four sub-regions. Each sub-region becomes a child node of the root node. The same judgment and quartering operation are recursively performed on each child node: the maximum value of the gradient residual values ​​of all pixels in the sub-region is calculated. If it is greater than the residual significance threshold, the quartering continues; otherwise, the splitting stops, and the child node becomes a leaf node. If the size of the sub-region reaches the preset minimum block size, the splitting stops regardless of whether the maximum value of the gradient residual is greater than the residual significance threshold, and the child node becomes a leaf node. The residual significance threshold is determined based on a preset multiple of the global mean of the gradient residual values ​​in the gradient residual plot. For example, the residual significance threshold can be set to 2.0 times the global mean of the gradient residual values. The minimum block size is determined based on consistency with the statistical block size in step S13. For example, the minimum block size can be set to 8 by 8 pixels.

[0116] See Figure 6 This is a schematic diagram of a sparse quadtree for gradient residual compensation provided in this application embodiment. As shown in the figure, the left side of the figure shows the spatial segmentation process of the gradient residual map, and the right side shows the corresponding quadtree hierarchical structure. In the gradient residual map on the left, the gray highlighted areas contain dense black dots, indicating that the gradient residual values ​​are significantly concentrated in this area, corresponding to the location where the fractal enhancement processing has significantly changed the gradient structure of this area; the remaining areas only have sparse light-colored dots, indicating that the gradient residuals are not significant. The quadtree on the right starts from the root node (corresponding to the entire image area) and splits into 4 child nodes layer by layer downwards. Sub-regions with insignificant gradient residuals stop splitting and are marked as empty nodes with dashed borders; sub-regions with significant gradient residuals continue splitting until the minimum block size is reached, generating effective leaf nodes marked with thick solid borders. Dashed arrows point from the highlighted areas of the gradient residual map on the left to the corresponding nodes on the right that continue splitting, marked "significant residuals, corresponding node splits". The bottom shows the field information stored in the effective leaf nodes, including the region location coordinates, region size, and the mean and maximum values ​​of the pixel gradient residuals within the region. When actually deployed on the AK612 uncooled infrared core, the sparse quadtree structure retains only effective leaf nodes in a small number of regions with significant gradient residuals. Leaf nodes corresponding to a large number of regions with insignificant gradient residuals are marked as empty nodes, thus not occupying storage and computing resources. This allows subsequent detailed compensation calculations to be concentrated in the regions that truly need correction, significantly reducing computational complexity and helping to meet the hardware constraints of the core's low power consumption.

[0117] S264: In each leaf node of the gradient residual compensation sparse quadtree, four fields are stored: region location coordinates (recording the top-left pixel coordinates of the sub-region corresponding to the leaf node in the gradient residual map), region size (recording the width and height of the sub-region), the mean of the pixel gradient residual values ​​within the region, and the maximum of the pixel gradient residual values ​​within the region. A preset compensation start threshold is set, which is determined based on the following: detailed compensation is only performed on regions where the mean gradient residual value is significantly higher than the global average level. For example, the compensation start threshold can be set to 1.0 times the global average gradient residual value. Only leaf nodes with a mean gradient residual value greater than the compensation start threshold within their respective leaf node regions are retained as valid leaf nodes, and the remaining leaf nodes are marked as empty nodes and no data is stored, forming a sparse storage structure, thus completing the construction of the gradient residual compensation sparse quadtree. The role of the sparse storage structure is that flat regions with small gradient residuals and edge regions with good processing effects do not require additional compensation. Marking the leaf nodes corresponding to these regions as empty nodes can significantly reduce storage space occupation and the computational load of subsequent step S3, and perform precise compensation only on regions with significant gradient residuals.

[0118] The core design idea of ​​step S2 is to apply differentiated enhancement strategies to pixels of different categories based on the pixel-level three-state labeled raster map generated in step S1, and to locate areas requiring further fine-tuning through gradient residual compensation sparse quadtree. In step S22, the gradient adaptive sharpening performed on edge-state pixels adopts a mechanism where the normalized edge strength value drives the sharpening gain coefficient, making the sharpening intensity proportional to the gradient response intensity of the edge itself. Strong edges receive strong sharpening, and weak edges receive weak sharpening, avoiding the problem of over-enhancing strong edges and under-enhancing weak edges caused by applying the same gain to all edge pixels in traditional Laplacian sharpening. In step S23, the gray-domain restricted bilateral smoothing performed on noisy-state pixels correlates the gray-domain standard deviation with the noise variance distribution map, allowing the smoothing intensity to adaptively adjust with the local noise level. This differs from the traditional bilateral filtering approach that uses a globally fixed gray-domain standard deviation, effectively solving the problem of insufficient or excessive smoothing caused by differences in noise levels in different regions. In step S24, the sharpening and smoothing weighted processing performed on transitional state pixels achieves a smooth transition from edge to noise through linear interpolation weights, avoiding visual discontinuities and detail gaps caused by hard switching between edge and noise state processing strategies. Step S26 constructs a gradient residual compensation sparse quadtree to accurately locate micro-detail regions that may be weakened in the fractal enhancement process, providing a spatial index and compensation intensity basis for detail compensation in step S3. Without the gradient residual analysis in step S26, step S3 would be unable to determine which regions lost detail information after fractal enhancement, and would have to perform detail compensation indiscriminately on the entire image. This would not only waste computing power but may also introduce unnecessary high-frequency interference into already well-processed areas. The sparse quadtree structure retains effective leaf nodes only in regions with significant gradient residuals, allowing subsequent compensation calculations to focus on the regions that truly need correction, significantly reducing computational complexity and helping to meet the low-power hardware constraints of the AK612 uncooled infrared core.

[0119] S3: Traverse all valid leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it on the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image to generate the global enhanced infrared image.

[0120] Further, step S3 includes:

[0121] S31: Traverse all valid leaf nodes in the gradient residual compensation sparse quadtree, and read the region location coordinates, region size, and average pixel gradient residual value stored in each valid leaf node. The number of valid leaf nodes depends on the effect of the fractal enhancement process: if the fractal enhancement process maintains a good gradient structure in most regions, the number of valid leaf nodes is small, and the subsequent compensation calculation is small; if the fractal enhancement process causes a large gradient structure shift in a specific region, the valid leaf nodes corresponding to that region will be retained to ensure the targeted nature of the compensation.

[0122] S32: For each valid leaf node, extract the corresponding pixel block in the reference infrared image based on the region location coordinates and region size, and perform high-pass filtering on the pixel block to extract micro-detail texture information.

[0123] Further, step S32 includes:

[0124] S321: Perform a 3x3 mean filter on the pixel block. Specifically, extract a 3x3 neighborhood for each pixel in the pixel block, calculate the arithmetic mean of the gray values ​​of the nine pixels in the neighborhood as the low-frequency component value of that pixel, and perform the above operation on all pixels in the pixel block to obtain a local low-frequency component map. Subtract the corresponding low-frequency component value in the local low-frequency component map from the gray value of each pixel in the pixel block in the reference infrared image to obtain the high-frequency detail residual block of the region. The values ​​in the high-frequency detail residual block represent the micro-detail texture information of the region in the reference infrared image, including subtle gray-level fluctuations, texture patterns, and weak target contour information. This information may be partially weakened by smoothing operations during the fractional enhancement process. See also Figure 7This figure illustrates the micro-detail texture extraction and compensation process provided in this application embodiment. As shown, the figure demonstrates the complete process of extracting micro-detail texture information from the pixel blocks corresponding to effective leaf nodes and generating a detail compensation amount that is then superimposed back onto the fractal enhancement image. The top represents an 8x8 grid of the effective leaf node pixel blocks, whose data is processed along two paths: the left path, after 3x3 mean filtering, yields a gray-filled local low-frequency component map, representing the smooth background information of that region; the main path retains the original pixel data downwards. The two paths converge at the subtraction node, where the local low-frequency component map is subtracted from the original pixel block to obtain a high-frequency detail residual block, represented by scattered black dots in the figure as the extracted weak texture information. The high-frequency detail residual block is multiplied by the compensation intensity coefficient provided on the right side via the multiplication node to generate a detail compensation block. Finally, the detail compensation block is superimposed onto the fractal enhancement image marked with a dashed box in the lower left corner via the addition node, outputting a detail compensation fusion image marked in bold in the lower right corner. In search and rescue and security scenarios using uncooled infrared imaging, the infrared radiation characteristics of distant, weak targets may appear as extremely subtle grayscale fluctuations. These subtle textures may be partially weakened during the noise smoothing process in step S23. This compensation mechanism accurately extracts the subtle texture information lost during smoothing from the baseline infrared image through high-pass filtering, and then scales it back to the fractional-enhanced image using a compensation intensity coefficient proportional to the mean gradient residual. This recovers the subtle textures lost during smoothing without amplifying the suppressed noise, thus improving the detection probability of small targets.

[0125] S322: Read the average pixel gradient residual within the region stored in the effective leaf node. Traverse all effective leaf nodes, read the average pixel gradient residual within their respective regions, and take the maximum value as the global average gradient residual maximum value. Divide the average pixel gradient residual within the effective leaf node by the global average gradient residual maximum value to obtain the compensation intensity coefficient of the effective leaf node. The calculation formula is:

[0126]

[0127] in This represents the mean pixel gradient residual within the region of the effective leaf node. This represents the maximum value of the global gradient residual mean. Compensation strength coefficient. The value range is a closed interval from 0 to 1. The larger the value, the greater the gradient structure shift in the region during the fractal enhancement process, and the stronger the detail compensation required.

[0128] S323: Multiply the value of each pixel in the high-frequency detail residual block by the compensation intensity factor. This yields the detail compensation block for the region corresponding to the effective leaf node. The value of each pixel in the detail compensation block is the amount of grayscale correction that needs to be superimposed onto the fractional-enhanced image for that pixel.

[0129] S33: Detail compensation blocks corresponding to the regions of all effective leaf nodes are superimposed onto the corresponding pixel positions of the fractal enhancement image according to their respective region coordinates. Specifically, for each effective leaf node, the value of each pixel in the detail compensation block is added to the grayscale value of the corresponding pixel in the fractal enhancement image. Pixels not covered by any effective leaf nodes retain their original grayscale values ​​from the fractal enhancement image, resulting in a detail compensation fused image. This detail compensation fused image, based on the fractal enhancement image, compensates for the micro-detail texture information lost due to fractal enhancement processing in regions with significant gradient residuals, while maintaining the effect of fractal enhancement processing in regions with insignificant gradient residuals.

[0130] S34: Perform grayscale dynamic range adaptive stretching on the detail-compensated fused image to obtain a globally enhanced infrared image.

[0131] Further, step S34 includes:

[0132] S341: A grayscale histogram of the detail-compensated fused image, wherein the grayscale histogram has grayscale values ​​on the horizontal axis and the number of pixels with that grayscale value appearing in the detail-compensated fused image on the vertical axis. A grayscale cumulative distribution function is calculated based on the grayscale histogram. The function value of the grayscale cumulative distribution function at a certain grayscale value is equal to the sum of the number of pixels corresponding to that grayscale value and all grayscale values ​​below it in the grayscale histogram, divided by the total number of pixels in the image. The grayscale value corresponding to when the grayscale cumulative distribution function value reaches a preset lower truncation percentage is determined and denoted as the lower stretching limit; the grayscale value corresponding to when the grayscale cumulative distribution function value reaches a preset upper truncation percentage is determined and denoted as the upper stretching limit. The lower truncation percentage is determined based on: truncating a very small number of pixels with extremely low grayscale values ​​to eliminate the effects of vignetting and low-response blind pixel residues. For example, the lower truncation percentage can be set to 0.5%. The determination of the upper truncation percentage is based on the following: truncating a very small number of pixels with extremely high grayscale values ​​to eliminate the influence of high-temperature hotspots and overexposed pixels. For example, the upper truncation percentage can be set to 99.5%. For example, assuming that after the grayscale histogram of the detail compensation fused image is calculated by the cumulative distribution function, the grayscale value corresponding to a cumulative distribution function value of 0.5% is 800, and the grayscale value corresponding to a cumulative distribution function value of 99.5% is 15000, then the lower limit of stretching is 800, and the upper limit of stretching is 15000.

[0133] S342: Perform linear stretching on the grayscale value of each pixel in the detail-compensated fused image. Specifically, let the grayscale value of a pixel in the detail-compensated fused image be... The lower limit of stretching is The maximum stretch value is The maximum value of the output grayscale range is The grayscale value of the stretched pixel The calculation formula is:

[0134]

[0135] like Less than the lower limit of stretching ,but Set to 0; if Greater than the upper limit of stretching ,but Set as The maximum value of the output grayscale range. The value depends on the output bit depth; for 8-bit grayscale output... The value is 255, for 14-bit grayscale output. The value is 16383. The stretched grayscale values ​​of all pixels are used to compose a globally enhanced infrared image. For example, assume a lower stretching limit value... The maximum stretch value is 800. The value is 15000, which represents the maximum value of the output grayscale range. The grayscale value of a pixel is 255. If it is 8000, then The purpose of the adaptive grayscale dynamic range stretching is that the effective grayscale range of the 14-bit grayscale image output by the uncooled infrared detector usually only occupies a portion of the full range, leaving a large number of grayscale levels unused, resulting in low image contrast and difficulty in discerning details. By using truncated linear stretching to map the effective grayscale range to the complete output grayscale range, the grayscale performance capability of the display device is fully utilized, thereby improving the overall contrast.

[0136] The core function of step S3 is to perform precise micro-detail compensation and global contrast optimization on the fractional enhancement result of step S2, ensuring that the weak texture information that may be lost during the fractional enhancement process is recovered, while adjusting the grayscale dynamic range of the image to a range suitable for display and subsequent processing. Steps S31 to S33 achieve targeted and precise detail compensation by traversing the effective leaf nodes of the gradient residual compensation sparse quadtree: compensation only occurs in areas where the gradient residual is significant, and the compensation intensity is proportional to the mean of the gradient residual, avoiding the noise amplification problem that may be introduced by uniform compensation across the entire image. If the detail compensation in step S3 is missing, although the bilateral smoothing processing of noisy pixels in step S2 effectively suppresses noise, the extremely weak texture information that may be mixed in the noisy region will be permanently lost. This weak texture information may correspond to the infrared radiation characteristics of distant weak targets in search and rescue scenarios, and its loss will directly affect the detection probability of small targets. The adaptive stretching of the grayscale dynamic range in step S34 automatically adapts to the grayscale range changes caused by the temperature distribution differences in different scenes through truncated linear stretching driven by the cumulative distribution function, without the need to manually adjust the display parameters for each scene. The synergistic relationship between steps S3 and S2 is as follows: Step S2 is responsible for performing differentiated enhancement and smoothing processing on the three types of pixels to resolve the core contradiction between edge sharpening and noise smoothing; Step S3 is responsible for detecting and compensating for the unavoidable loss of micro-details during the processing of Step S2 and optimizing global contrast. The two steps constitute a complete detail enhancement chain of enhancement-compensation-stretching. If only the fractional enhancement of Step S2 is performed and the detail compensation and grayscale stretching of Step S3 are skipped, the fractional enhanced image, although improved in terms of edge sharpness and noise suppression, will have limited detail expression and contrast of the final image due to the loss of weak texture and insufficient utilization of grayscale range.

[0137] S4: Performs a fixed-point parallel processing pipeline deployment for the global enhanced infrared image, targeting the uncooled infrared camera module. It performs row buffer block scheduling according to the camera module's imaging specifications, realizes row-by-row pipelined processing in the field-programmable gate array, and outputs the final enhanced infrared image.

[0138] Further, step S4 includes:

[0139] S41: Convert all floating-point operations involved in steps S1 to S3 into fixed-point operations to obtain the set of parameters for the full-process fixed-point algorithm.

[0140] Further, step S41 includes:

[0141] S411: The fixed-point bit width for the full-process fixed-point calculation is determined to be a 16-bit signed fixed-point number, including 1 sign bit, 9 integer bits, and 6 decimal bits. The representable numerical range is from -512 to +511.984375, with a precision of 0.015625. The determination of the fixed-point bit width is based on the following: The maximum 14-bit grayscale value output by the uncooled infrared detector is 16383. After gradient calculation, the maximum absolute value of the gradient component does not exceed 7 times 16383, which equals 114681. However, in actual scenarios, the grayscale difference between adjacent pixels is usually much smaller than the full scale. The representation range of 9 integer bits (512) is sufficient to cover the actual value range of intermediate variables such as the multi-scale gradient amplitude in step S12 and the sharpening gain coefficient in step S22. The precision of 6 decimal bits (0.015625) is sufficient to maintain the calculation accuracy of variables requiring decimal precision, such as the bilateral smoothing weight in step S23 and the linear stretching ratio in step S34. In the fixed-point normalization process, the square root operation involved in gradient magnitude calculation in step S12 is replaced by a piecewise linear lookup table operation. The piecewise linear lookup table pre-calculates the square roots of integer values ​​in the range of 0 to 512 and stores them in on-chip read-only memory. For non-integer inputs, an approximate result is obtained by linear interpolation of the square roots of two adjacent integer values. The multiplication operation involved in adaptive dual-threshold calculation in step S14 is replaced by a shift-accumulation operation. Specifically, after representing the high threshold coefficient and low threshold coefficient with fixed-point numbers, the multiplication operation with the global noise mean square error is implemented by shifting the global noise mean square error to the left by the corresponding number of bits and then accumulating. The division operation involved in sharpening gain coefficient calculation in step S22 is replaced by a reciprocal multiplication operation. Specifically, the 16-bit fixed-point reciprocal of the maximum edge gradient magnitude is pre-calculated and stored. During normalization calculation, the edge gradient magnitude is multiplied by this reciprocal instead of division. The exponential function operations involved in the spatial domain weight and grayscale domain weight calculations in step S23 are replaced with piecewise linear lookup table operations. The piecewise linear lookup table pre-calculates and stores the function values ​​of the Gaussian function at 256 equally spaced sampling points within the range of 0 to 8. Similarly, the division operations involved in the compensation intensity coefficient calculations in step S32 are replaced with reciprocal multiplication operations. The division operations involved in the linear stretching calculations in step S34 are replaced with reciprocal multiplication operations, and the 16-bit fixed-point reciprocal of the difference between the upper and lower stretching limits is pre-calculated. All the fixed-point coefficients, lookup tables, and operation rules described above are summarized into a complete fixed-point algorithm parameter set.

[0142] S412: Verify the fixed-point accuracy of the full-process fixed-point algorithm parameter set. Specifically, select 10 reference infrared images covering four typical scenarios: indoor monitoring, outdoor security, seascape background, and vehicle night vision. Perform all calculations from steps S1 to S3 using both the floating-point algorithm and the full-process fixed-point algorithm parameter set. For each image, calculate the average absolute value of the pixel-by-pixel grayscale difference between the floating-point result and the fixed-point result, and record this as the average fixed-point grayscale deviation. If the average fixed-point grayscale deviation is less than a preset fixed-point accuracy tolerance threshold, the full-process fixed-point algorithm parameter set passes verification. The fixed-point accuracy tolerance threshold is determined based on the following: in 8-bit grayscale output, a difference of one grayscale level is almost imperceptible to the human eye; therefore, the fixed-point accuracy tolerance threshold should be much smaller than one grayscale level. For example, the fixed-point accuracy tolerance threshold can be set to 0.5 grayscale levels.

[0143] S42: Design a line buffer block scheduling scheme for the AK612 uncooled infrared camera module with an imaging specification of 640 x 512.

[0144] Further, step S42 includes:

[0145] S421: Based on the requirements of a 7x7 maximum gradient calculation window in step S12 and a 5x5 bilateral smoothing window in step S23, the row cache depth is determined to be 7 rows. The row cache depth is determined as follows: a 7x7 window requires that pixel data in the three rows above and below the current pixel be available simultaneously; a 5x5 window requires that pixel data in the two rows above and below the current pixel be available simultaneously. The larger of these two values ​​is taken as 3 rows, plus the current row, requiring a total of 7 rows of data to be available on-chip simultaneously. A 7-row by 640-column row cache region is allocated in the Field Programmable Gate Array (FPGA). This row cache region is implemented using block random access memory (BRAM), with each memory cell being 16 bits wide to match the fixed-point bit width, occupying a total of 7 x 640 x 16 = 71680 bits of on-chip storage resources.

[0146] S422: The reference infrared image is written line by line sequentially to the line buffer area, using a first-in-first-out (FIFO) circular overwrite method to manage the line buffer area. Specifically, the line buffer area maintains a write pointer, initially pointing to the storage location of the first line. After each new line of data is written, the write pointer moves to the next storage location. When the write pointer exceeds the 7th line, it wraps back to the 1st line to overwrite the earliest written line data. After each new line of data is written, it is determined whether the line buffer area has accumulated 7 lines of valid data. If it has, the 3rd line above the current written line is used as the center line. A 7x640 pixel block centered on the center line is extracted from the line buffer area. The full-process fixed-point calculation of steps S1 to S3 is performed on each pixel in the center line of this pixel block, and the final enhanced grayscale value of the 640 pixels in the center line is output. For pixels in the first 3 lines and the last 3 lines of the image that cannot form a 7x7 window due to incomplete neighborhoods, a mirror filling method is used to fill in the missing lines, that is, the line data within the boundary is mirrored and copied to the missing line position outside the boundary.

[0147] S43: Deploys 3 levels of parallel processing units in a pipelined architecture within a field-programmable gate array.

[0148] Further, step S43 includes:

[0149] S431: The first-level processing unit receives a 7x640 pixel block centered on the center row in the row buffer region, and performs multi-scale gradient calculation in step S12, noise variance estimation in step S13, and tri-state classification labeling in step S14. The first-level processing unit calculates the horizontal and vertical gradient components in parallel for each pixel in the center row across three scale windows. The gradient calculations at the three scales are performed simultaneously by three independent difference operation submodules. After the calculations are completed, the multi-scale gradient magnitude and multi-scale gradient direction angle are obtained by weighting according to the scale weights. Simultaneously, the first-level processing unit calculates the grayscale variance of the 8x8 statistical block containing the center row pixels as a local noise variance estimate. Based on the multi-scale gradient magnitude, multi-scale gradient direction angle, and local noise variance estimate, the first-level processing unit performs improved non-maximum suppression and adaptive double threshold comparison, outputting the tri-state labeling result and multi-scale gradient magnitude for each pixel in the center row.

[0150] S432: The second-level processing unit receives the tri-state annotation results and multi-scale gradient magnitudes of each pixel in the center row from the first-level processing unit. Based on the tri-state annotation results, it performs corresponding enhancement processing on each pixel in the center row. Specifically, for pixels with tri-state annotation results in an edge state, it performs gradient adaptive sharpening in step S22; for pixels with tri-state annotation results in a noisy state, it performs grayscale domain-constrained bilateral smoothing in step S23; and for pixels with tri-state annotation results in a transitional state, it performs a sharpening-smoothing hybrid weighted processing in step S24. The second-level processing unit internally sets up three parallel data paths corresponding to the processing logic of the three annotation states, and uses the tri-state annotation results to control the multiplexer to route the data of the current pixel to the corresponding processing path. The second-level processing unit outputs the tri-state enhanced grayscale value for each pixel in the center row.

[0151] S433: The third-level processing unit receives the fractional enhanced grayscale value of each pixel in the center row output by the second-level processing unit, and performs detail compensation in step S32 and adaptive grayscale dynamic range stretching in step S34. Specifically, the third-level processing unit determines whether the current pixel is located within the region corresponding to a certain effective leaf node based on the region position coordinates of the effective leaf nodes in the gradient residual compensation sparse quadtree. If so, it reads the pixel data of the region where the pixel is located from the reference infrared image in the row buffer region, performs high-pass filtering, and calculates the detail compensation amount to be superimposed on the fractional enhanced grayscale value. Otherwise, the fractional enhanced grayscale value remains unchanged. Then, linear stretching is performed on the superimposed grayscale value to output the final enhanced grayscale value of each pixel in the center row.

[0152] S434: The three-stage processing units operate simultaneously in a pipeline manner, with the first-stage processing unit processing the... When adding two rows of data, the second-level processing unit simultaneously processes the first row. Add one row of data, and the third-level processing unit will process the data simultaneously. The data is processed row by row in parallel pipeline. The physical meaning of the pipeline method is: when the first-stage processing unit completes the first row of data... After adding a row of tri-state annotations and passing the result to the second-level processing unit, the first-level processing unit immediately begins processing the... Add two lines of input data; there's no need to wait for the second and third level processing units to complete processing the first line. Add 1 row and the first Line processing. The steady-state throughput of the 3-stage pipeline processes one pixel per clock cycle. One line takes 640 clock cycles to process, and 512 lines require 640 x 512 = 327,680 clock cycles, plus the pipeline startup delay of two lines. The AK612 uncooled infrared sensor has a frame rate of 25 frames per second, with each frame processing time not exceeding 40 milliseconds. At a 100 MHz master clock frequency, 327,680 + 1280 = 328,960 clock cycles take approximately 3.29 milliseconds, far less than the 40-millisecond frame interval, meeting real-time requirements.

[0153] See Figure 8 This is a schematic diagram of a three-stage pipelined parallel processing provided in an embodiment of this application. As shown in the figure, the horizontal axis of the diagram shows the pipeline architecture in which three processing units work in parallel for different rows of data at the same time. The first-stage processing unit performs gradient calculation and tri-state classification, and is currently processing the n+2th row of data; the second-stage processing unit performs state differentiation enhancement, and is currently processing the n+1th row of data at the same time; the third-stage processing unit performs detail compensation and grayscale stretching, and is currently processing the nth row of data at the same time. The three stages are marked with vertical dashed lines indicating "at the same time" to indicate their parallel relationship, and the connecting lines between stages marked "transferring annotation results" and "transferring enhanced grayscale values" to indicate the direction of data flow between adjacent stages. The row sequence on the right side of each processing unit is represented by squares of different gray levels to indicate completed, currently being processed, and pending rows of data, and dashed boxes to indicate subsequent rows of data that have not yet entered that stage. Under the constraint of a real-time imaging frame rate of 25 frames per second for the AK612 uncooled infrared sensor, this three-stage pipeline architecture enables the three processing stages of multi-scale gradient calculation and three-state classification, state differentiation enhancement, detail compensation and grayscale stretching to be executed in parallel for data from different rows. The steady-state throughput reaches processing 1 pixel per clock cycle. At a main clock frequency of 100 MHz, the processing time per frame is about 3.29 milliseconds, which is much less than the 40 millisecond frame interval, ensuring the real-time running capability of all algorithms on the embedded hardware platform.

[0154] S44: Combine the final enhanced grayscale values ​​of all rows output by the third-level processing unit in row order to output the final enhanced infrared image. The resolution of the final enhanced infrared image is 640 x 512, which meets the real-time output requirements of the uncooled infrared camera module. The final enhanced infrared image is output to an external display device or a back-end image analysis system via the video interface of the AK612 uncooled infrared camera module according to a preset video format, for target observation and identification in application scenarios such as monitoring, security, search and rescue, vehicle-mounted and handheld detection.

[0155] The core function of step S4 is to convert all algorithms designed in steps S1 to S3 from floating-point arithmetic to fixed-point arithmetic, and to implement row-by-row pipelined parallel processing on the field-programmable gate array (FPGA) hardware platform. This enables the entire edge-preserving infrared image digital detail enhancement method to run in real time under the hardware resource constraints of the AK612 uncooled infrared chip. In step S411, the fixed-point arithmetic replaces all floating-point multiplication with shift-accumulation, all floating-point division with reciprocal multiplication or lookup table operations, and all transcendental functions with piecewise linear lookup table operations. This eliminates the need for floating-point arithmetic units, allowing the algorithm to be executed efficiently on FPGA chips without floating-point processing capabilities, while significantly reducing power consumption. The row cache block scheduling scheme in step S42 transforms full-frame processing into row-by-row processing. Only a 7-row by 640-column row cache area needs to be maintained on-chip to meet the data requirements of the maximum window size, avoiding the huge storage overhead of storing the entire frame image on-chip. The three-stage pipeline architecture in step S43 enables the three processing stages—multi-scale gradient calculation and three-state classification, state differentiation enhancement, and detail compensation and grayscale stretching—to be executed in parallel for data from different rows. This compresses the processing latency of a single frame image from the serial time of the three stages to the equivalent time of a single stage under pipeline steady-state conditions. This significantly reduces the processing time per frame to the frame interval, meeting the AK612 uncooled infrared sensor's real-time imaging frame rate requirement of 25 frames per second and the low power consumption constraint of a stable power consumption not exceeding 2 watts. Without the fixed-pointing and pipeline deployment in step S4, the digital detail enhancement algorithms designed in steps S1 to S3 would only be able to execute offline on a general-purpose processor using floating-point operations, unable to be embedded in the AK612 uncooled infrared sensor for online real-time enhancement, thus greatly diminishing the algorithm's engineering application value. The synergistic relationship between step S4 and steps S1 to S3 is as follows: Steps S1 to S3 solve the core technical contradiction between edge sharpening and noise smoothing at the algorithm level and realize micro-detail compensation and global contrast optimization. Step S4 transforms the algorithm into an engineering solution that can run in real time on an embedded platform from the hardware implementation level. Together, they constitute a complete technical link from algorithm design to hardware deployment, enabling the edge-preserving infrared image digital detail enhancement method to have a feasible engineering implementation capability in the actual application scenarios such as monitoring, security, search and rescue, vehicle and handheld detection targeted by the AK612 uncooled infrared core.

[0156] For example, taking a 640x512 resolution 14-bit grayscale raw infrared image acquired by the AK612 uncooled infrared detector in an outdoor security scenario as an example, the complete workflow is as follows: In step S111, after the uncooled infrared detector acquires the raw infrared image, step S112 uses the factory-calibrated gain correction coefficient and offset correction coefficient to perform two-point non-uniform correction on each pixel. In step S113, median replacement is performed on the bad pixel according to the blind pixel position mapping table and the dynamic blind pixel detection results to obtain the reference infrared image. Step S121 calculates the horizontal and vertical gradient components of a pixel in the reference infrared image under 3x3, 5x5, and 7x7 windows respectively. Step S122 calculates the single-scale gradient magnitude and single-scale gradient direction angle at the three scales. Step S123 performs weighted fusion according to scale weights of 0.5, 0.3, and 0.2 to obtain the multi-scale gradient magnitude and multi-scale gradient direction angle. Assuming that the single-scale gradient magnitude of a pixel at the three scales is 150, 130, and 110 respectively, the multi-scale gradient magnitude is 150 x 0.5 + 130 x 0.3 + 110 x 0.2 = 136. Steps S131 to S133 divide the reference infrared image into 5120 8x8 statistical blocks and calculate the gray-level variance and bilinear interpolation to obtain the noise variance distribution map. Step S141 performs improved non-maximum suppression to obtain the suppressed gradient magnitude map. Step S142 assumes the global noise mean square error is 25, the high threshold coefficient is 4.0, and the low threshold coefficient is 1.5, then the high threshold is 100 and the low threshold is 37.5. Step S143 labels pixels with a suppressed gradient magnitude greater than or equal to 100 as edge states, less than 37.5 as noise states, and those in between as transition states. Step S144 constructs a pixel-level three-state labeled raster map after performing connectivity checks. Steps S21 to S25 perform gradient adaptive sharpening, gray-scale domain restricted bilateral smoothing, and a weighted blend of sharpening and smoothing on the three types of pixels to obtain a state-enhanced image. Step S26 calculates the gradient residual map and constructs a gradient residual compensated sparse quadtree. Steps S31 to S33 perform high-pass filtering and detail compensation on the regions corresponding to the effective leaf nodes to obtain a detail-compensated fused image. Step S341 assumes a lower truncation percentage of 0.5% and an upper truncation percentage of 99.5%, determining a lower stretching limit of 800 and an upper stretching limit of 15000. Step S342 performs linear stretching on a pixel with a grayscale value of 8000. After stretching, the grayscale value is (8000 - 800) divided by (15000 - 800) multiplied by 255, approximately equal to 129, yielding a globally enhanced infrared image. Steps S411 to S412 convert all operations to 16-bit fixed-point operations and verify accuracy. Step S421 allocates a 7-row by 640-column row buffer area in a field-programmable gate array. Steps S431 to S434 process row by row in a 3-stage pipeline, with each frame taking approximately 3.29 milliseconds at a 100 MHz master clock.Step S44 outputs a final enhanced infrared image with a resolution of 640 x 512, which is transmitted to the monitoring display terminal via a video interface.

[0157] Example 2:

[0158] This embodiment, based on Embodiment 1, provides an edge-preserving infrared image digital detail enhancement system, such as... Figure 9 As shown, it includes:

[0159] The reference image preprocessing and tri-state annotation module is used to perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain the reference infrared image, perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image, and construct a pixel-level tri-state annotation raster image.

[0160] The morphological differentiation enhancement and gradient residual analysis module is used to perform morphological differentiation enhancement processing on the reference infrared image based on the pixel-level three-state labeled raster image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-limited bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a morphological enhancement image. The morphological enhancement image and the reference infrared image are then compared pixel by pixel to calculate the gradient residual and construct a gradient residual compensation sparse quadtree.

[0161] The detail compensation and grayscale stretching module is used to traverse all effective leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it on the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image, and generate a global enhanced infrared image.

[0162] Fixed-point parallel processing deployment module: used to perform fixed-point parallel processing pipeline deployment for uncooled infrared cores on global enhanced infrared images, perform row buffer block scheduling according to the core imaging specifications, realize row-by-row pipelined processing in field programmable gate array and output the final enhanced infrared image.

Claims

1. A method for enhancing digital details in infrared images based on edge preservation, characterized in that, The method includes: S1: Perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain the reference infrared image. Perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image to construct a pixel-level three-state labeled raster image. S2: Based on the pixel-level three-state labeled raster image, perform state-differential enhancement processing on the reference infrared image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-limited bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a state-enhanced image. Perform pixel-by-pixel gradient residual calculation on the state-enhanced image and the reference infrared image to construct a gradient residual compensation sparse quadtree. S3: Traverse all effective leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it onto the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image to generate the global enhanced infrared image. S4: Performs a fixed-point parallel processing pipeline deployment for the global enhanced infrared image, targeting the uncooled infrared camera module. It performs row buffer block scheduling according to the camera module's imaging specifications, realizes row-by-row pipelined processing in the field-programmable gate array, and outputs the final enhanced infrared image.

2. The infrared image digital detail enhancement method based on edge preservation according to claim 1, characterized in that, The steps for constructing a pixel-level tri-state labeled raster image include: The raw infrared image acquired by the uncooled infrared detector is obtained. Non-uniformity correction is performed on the raw infrared image to eliminate the non-uniformity of the detector response. Then, blind pixel replacement is performed on the non-uniformly corrected image to eliminate bad pixel defects, thus obtaining the reference infrared image. Multi-scale gradient calculations are performed on the reference infrared image to obtain a multi-scale gradient magnitude map and a multi-scale gradient direction map; Local noise statistical modeling is performed on the reference infrared image to obtain the noise variance distribution map; Based on multi-scale gradient magnitude map, multi-scale gradient direction map and noise variance distribution map, three-state classification and labeling are performed on each pixel in the reference infrared image to construct a pixel-level three-state labeled raster map.

3. The edge-preserving infrared image digital detail enhancement method according to claim 2, characterized in that, The steps for obtaining the multi-scale gradient magnitude map and the multi-scale gradient direction map include: Three gradient calculation windows are set up, namely 3x3 window, 5x5 window and 7x7 window. Under each scale window, the gray level difference of each pixel in the reference infrared image is calculated along the horizontal and vertical directions to obtain the horizontal gradient component and the vertical gradient component at that scale. For each pixel at each scale, the gradient magnitude and gradient direction angle are calculated based on the horizontal and vertical gradient components, resulting in a single-scale gradient magnitude map and a single-scale gradient direction map at three scales. The single-scale gradient magnitude maps at three scales are weighted and fused according to scale weights. The scale weights are allocated according to the principle that the weight of the small-scale window is greater than the weight of the large-scale window. The single-scale gradient direction maps at three scales are weighted and fused according to the same scale weights to obtain a multi-scale gradient direction map.

4. The edge-preserving infrared image digital detail enhancement method according to claim 2, characterized in that, The step of obtaining the noise variance distribution map includes: The baseline infrared image is divided into non-overlapping statistical blocks of fixed size, each block being 8 by 8 pixels. For each statistical block, calculate the mean and variance of the gray values ​​of 64 pixels within the block, and assign the gray value variance to all 64 pixels within the statistical block as an estimate of the local noise variance. Bilinear interpolation is performed on the local noise variance estimates of all statistical blocks to generate a noise variance distribution map with the same resolution as the baseline infrared image.

5. The edge-preserving infrared image digital detail enhancement method according to claim 2, characterized in that, The steps for constructing a pixel-level tri-state labeled raster image include: An improved nonmaximum suppression is performed on the multi-scale gradient magnitude map to obtain the suppressed gradient magnitude map; Calculate the adaptive dual threshold based on the noise variance distribution map; Based on the suppressed gradient magnitude map and adaptive dual thresholds, tri-state classification labeling is performed on each pixel in the benchmark infrared image; A connectivity test is performed on all pixels labeled as transitional states, and the final labeling results of all pixels are written into a raster structure with the same resolution as the baseline infrared image to form a pixel-level three-state labeled raster map.

6. The method for enhancing digital details of infrared images based on edge preservation according to claim 1, characterized in that, The steps for constructing the gradient residual compensation sparse quadtree include: Traverse the pixel-level three-state labeled raster image to extract the set of edge state pixel locations, the set of noise state pixel locations, and the set of transition state pixel locations; Perform gradient adaptive sharpening on each pixel in the set of edge-state pixel locations; Perform grayscale-limited bilateral smoothing on each pixel in the set of noisy pixel locations; Perform sharpening and smoothing blending weighted processing on each pixel in the transition state pixel location set; The edge-sharpened gray values ​​of edge-state pixels, the noise-smoothed gray values ​​of noise-state pixels, and the transition-mixed gray values ​​of transition-state pixels are written back into the same image according to their respective pixel position coordinates to obtain a state-enhanced image. A gradient residual compensated sparse quadtree is constructed based on the fractal enhanced image and the reference infrared image.

7. The infrared image digital detail enhancement method based on edge preservation according to claim 6, characterized in that, The step of performing gradient adaptive sharpening processing on each pixel in the set of edge state pixel locations includes: For the current pixel in the edge state pixel location set, read the multi-scale gradient magnitude stored in the corresponding grid cell of the pixel in the pixel-level three-state labeled raster map, traverse the multi-scale gradient magnitude corresponding to all pixels in the edge state pixel location set, take the maximum value among them as the maximum edge gradient magnitude, divide the edge gradient magnitude of the pixel by the maximum edge gradient magnitude to obtain the normalized edge intensity value. The sharpening gain coefficient for this pixel is calculated based on the normalized edge intensity value; For the edge-state pixel, extract a 3x3 neighborhood in the reference infrared image, calculate the Laplacian operator response value, multiply the Laplacian operator response value by the sharpening gain coefficient, and then superimpose it onto the original grayscale value of the pixel in the reference infrared image to obtain the edge-sharpened grayscale value of the pixel.

8. The edge-preserving infrared image digital detail enhancement method according to claim 6, characterized in that, The step of performing grayscale-limited bilateral smoothing on each pixel in the set of noisy pixel locations includes: For the noisy pixel, a 5x5 neighborhood is extracted from the reference infrared image. The spatial distance between each pixel in the neighborhood and the center pixel is calculated. The spatial domain weight is calculated based on the spatial distance and the preset spatial domain standard deviation. Calculate the absolute value of the difference between the gray value of each pixel in the neighborhood and the gray value of the center pixel. Read the noise variance estimate at the position of the center pixel in the noise variance distribution map. Take the square root of the noise variance estimate to obtain the gray-domain standard deviation. Calculate the gray-domain weight based on the absolute value of the gray value difference and the gray-domain standard deviation. Multiply the gray value of each pixel in the neighborhood by the product of the spatial domain weight and the gray-level domain weight, sum the weighted gray values ​​of all pixels, and then divide by the sum of the products of the spatial domain weight and the gray-level domain weight of all pixels to obtain the noise-smoothed gray value of that pixel.

9. The method for enhancing digital details of infrared images based on edge preservation according to claim 1, characterized in that, The step of generating a globally enhanced infrared image includes: Traverse all valid leaf nodes in the gradient residual compensation sparse quadtree and read the region location coordinates, region size and average pixel gradient residual value stored in each valid leaf node. For each valid leaf node, the corresponding pixel block is extracted from the reference infrared image based on the region location coordinates and region size, and high-pass filtering is performed on the pixel block to extract micro-detail texture information; The detail compensation blocks corresponding to the regions of all effective leaf nodes are superimposed onto the corresponding pixel positions of the fractal enhancement image according to their respective region position coordinates to obtain the detail compensation fused image; The grayscale dynamic range adaptive stretching is performed on the detail-compensated fused image to obtain a globally enhanced infrared image.

10. An edge-preserving infrared image digital detail enhancement system, used to implement the edge-preserving infrared image digital detail enhancement method according to any one of claims 1-9, characterized in that, The system includes: The reference image preprocessing and tri-state annotation module is used to perform non-uniform correction and blind pixel replacement on the raw infrared image acquired by the uncooled infrared detector to obtain the reference infrared image, perform multi-scale gradient analysis and noise statistical modeling on the reference infrared image, and construct a pixel-level tri-state annotation raster image. The morphological differentiation enhancement and gradient residual analysis module is used to perform morphological differentiation enhancement processing on the reference infrared image based on the pixel-level three-state labeled raster image, perform gradient adaptive sharpening on edge state pixels, perform gray-scale domain-limited bilateral smoothing on noise state pixels, and perform sharpening and smoothing hybrid weighted processing on transition state pixels to obtain a morphological enhancement image. The morphological enhancement image and the reference infrared image are then compared pixel by pixel to calculate the gradient residual and construct a gradient residual compensation sparse quadtree. The detail compensation and grayscale stretching module is used to traverse all effective leaf nodes of the gradient residual compensation sparse quadtree, extract the micro-detail texture information of the corresponding region in the baseline infrared image, generate the detail compensation amount and superimpose it on the fractal enhancement image to obtain the detail compensation fused image, perform grayscale dynamic range adaptive stretching on the detail compensation fused image, and generate a global enhanced infrared image. Fixed-point parallel processing deployment module: used to perform fixed-point parallel processing pipeline deployment for uncooled infrared cores on global enhanced infrared images, perform row buffer block scheduling according to the core imaging specifications, realize row-by-row pipelined processing in field programmable gate array and output the final enhanced infrared image.

Images