Infrared image-oriented multi-dimensional complementary definition evaluation method

Abstract

The invention discloses a multi-dimensional complementary definition evaluation method for an infrared image. The method comprises the following steps: acquiring an infrared image sequence, an equipment parameter set and environment observation data; performing basic calibration on the infrared image sequence to generate a basic calibration image sequence; executing physical parameter estimation and normalized modeling based on the basic calibration image sequence, the equipment parameter set and the environment observation data to obtain a normalized definition sub-index set and a scene description vector; and according to the normalized definition sub-index set and the scene description vector, performing multi-dimensional complementary definition construction and unimodal enhancement, and generating a unimodal enhanced definition curve and a real-time definition value. The invention also includes performing a focus search using a curve, detecting a side lobe risk in the search, and dynamically writing back parameters to suppress side lobes. According to the method, the scale consistency and the anti-interference stability are improved, the form is optimized, and the convergence is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of sharpness evaluation, and in particular to a multi-dimension complementary sharpness evaluation method for infrared images. BACKGROUND

[0002] Infrared imaging technology is essential in the fields of industry and auxiliary driving due to its all-weather and anti-interference advantages. Auto-focusing is the core function of an infrared system, and its speed and robustness directly determine the system's ability to obtain clear thermal images, which is a key technical prerequisite for ensuring system effectiveness.

[0003] Currently, passive auto-focusing is the mainstream solution for infrared systems. This technology usually involves capturing a series of images at different focal length positions and calculating a predefined sharpness evaluation function for each image. These functions mainly include operators based on image gradients (such as Tenengrad), operators based on statistical variance (such as Laplacian), or operators based on frequency domain analysis (such as Fourier high-frequency energy). The system uses search strategies such as hill climbing to find the focal point position that maximizes the function value.

[0004] However, the above-mentioned method relying on empirical operators has two major technical bottlenecks in practical applications: the scale variability of sharpness measurement and the morphological non-ideality. First, the scale variability is due to the extreme sensitivity of the measurement value to non-focusing factors (such as automatic gain control AGC). When scene changes cause AGC to switch segments, the image contrast changes dramatically, causing the gradient or variance-based measurement value to produce a large false peak unrelated to the focal point, which often causes the search algorithm to lock the wrong position. The morphological non-ideality is due to the fact that these empirical operators do not consider the physical characteristics of infrared imaging (such as thermal diffusion), resulting in a clear curve with strong side lobes or wide flat tops in low-texture or high-noise scenes. Non-ideal morphology makes traditional search algorithms prone to local optimization (side lobes) or oscillation on the peak top, making it impossible to converge. SUMMARY

[0005] The application aims to provide a multi-dimension complementary sharpness evaluation method for infrared images to solve the above-mentioned problems existing in the prior art.

[0006] Technical solution, according to one aspect of the application, a multi-dimension complementary sharpness evaluation method for infrared images, comprising:

[0007] Obtaining an infrared image sequence, a device parameter set, and environmental observation data;

[0008] Performing basic calibration on the infrared image sequence to generate a basic calibration image sequence;

[0009] Based on the basic calibration image sequence, the device parameter set and the environment observation data, physical parameter estimation and normalized modeling are performed to obtain a normalized sharpness sub-index set and a scene description vector;

[0010] According to the normalized sharpness sub-index set and the scene description vector, multi-dimensional complementary sharpness construction and unimodality enhancement are performed to output a unimodal enhanced sharpness curve and a real-time sharpness value;

[0011] Among them, the multi-dimensional complementary sharpness construction and unimodality enhancement include:

[0012] Construct a physically consistent sharpness index;

[0013] Construct a hot core multi-scale tensor sharpness index;

[0014] Perform physical gating fusion on the two types of indexes to generate a fused sharpness;

[0015] Apply unimodal enhancement transformation to the fused sharpness and perform online self-calibration to output a unimodal enhanced sharpness curve and a real-time sharpness value.

[0016] According to one aspect of the present application, performing physical gating fusion includes:

[0017] Apply virtual defocus perturbation and noise injection to the basic calibration image sequence to evaluate the unimodality prior and noise sensitivity of the two types of indexes, and construct their respective free energy metrics;

[0018] Extract the scene equivalent temperature from the scene description vector;

[0019] According to the free energy metric and the scene equivalent temperature, calculate the physical gating weight through the Boltzmann distribution;

[0020] Use the physical gating weight to weight the two types of indexes to generate a fused sharpness.

[0021] According to one aspect of the present application, performing online self-calibration includes:

[0022] Perform a short window micro-scan around the current focus to collect a local unimodal enhanced sharpness sampling sequence;

[0023] Calculate the unimodality index of the local sampling sequence by determining the minimum value of the second derivative of the logarithmic curve of the sequence, and combining the ratio of the secondary peak height to the main peak height;

[0024] Maximize the unimodality index as the target, and update the parameters of the unimodal enhancement transformation or the weights of the physical gating fusion online;

[0025] Based on the updated parameters or weights, generate the final unimodal enhanced sharpness curve and real-time sharpness value.

[0026] According to an aspect of the present application, a physical-consistent sharpness metric is constructed, comprising:

[0027] A frequency-domain weight function is constructed according to a composite modulation transfer function comprising optical defocus variance and thermal diffusion equivalent time;

[0028] An amplitude spectrum is calculated for the base calibration image sequence, and a frequency-domain weighted integral is performed using the weight function to obtain a raw physical-consistent sharpness;

[0029] A set of normalization transformers is obtained from the physical parameter estimation and normalization modeling step, and is invoked to perform scale alignment on the raw physical-consistent sharpness.

[0030] According to an aspect of the present application, a thermal kernel multi-scale tensor sharpness metric is constructed, comprising:

[0031] A thermal kernel scale set is determined according to a scene description vector;

[0032] A three-dimensional tensor is constructed by convolving the base calibration image sequence using the thermal kernel scale set;

[0033] Tensor decomposition is performed on the three-dimensional tensor to extract a direction energy singular value and a scale mode factor matrix;

[0034] Based on the direction energy singular value and the scale mode factor matrix, a direction energy concentration and a scale energy concentration are calculated to form a raw thermal kernel multi-scale tensor sharpness;

[0035] A set of normalization transformers is obtained from the physical parameter estimation and normalization modeling step, and is invoked to perform scale alignment on the raw thermal kernel multi-scale tensor sharpness.

[0036] According to an aspect of the present application, the method further comprises:

[0037] A focus search and control loop is performed using the unimodal enhanced sharpness curve and the real-time sharpness value to determine the optimal focus position;

[0038] The focus search and control loop comprises performing a three-point fitting, performing a side lobe detection, and generating a focus drive sequence;

[0039] The side lobe detection comprises:

[0040] During the focus search process, a side lobe risk is determined based on a secondary peak to primary peak ratio or curvature inconsistency;

[0041] In response to the side lobe risk, a side lobe suppression parameter increment is generated;

[0042] The side lobe suppression parameter increment is written back to the multi-dimension complementary sharpness construction and unimodal enhancement step to dynamically change the shape of the unimodal enhanced sharpness curve in subsequent sharpness calculations.

[0043] According to an aspect of the present application, performing three-point fitting, comprising:

[0044] Collecting real-time sharpness values corresponding to focus point positions in search interval;

[0045] Fitting a parabola to the focus point positions and real-time sharpness values to obtain candidate focus point positions and local curvature estimates;

[0046] According to the local curvature estimates, setting the step size suggestion for the next movement to achieve curvature adaptive step size.

[0047] According to an aspect of the present application, generating a focus driving sequence, comprising:

[0048] Obtaining a lens mechanism model in the device parameter set;

[0049] According to the candidate focus point position or the step size suggestion, generating an initial driving sequence using trajectory planning with jerk constraint;

[0050] Combining the lens mechanism model, performing reverse gap compensation and feedforward friction compensation on the initial driving sequence to generate the final focus driving sequence.

[0051] According to an aspect of the present application, performing physical parameter estimation and normalized modeling, comprising:

[0052] Estimating noise variance, automatic gain function and gain energy factor, and bit depth;

[0053] Estimating the scene equivalent temperature corresponding to the environmental observation data;

[0054] Constructing a closed normalized mapping to generate a set of normalized sharpness sub-indices, and aggregating the scene equivalent temperature to generate a scene description vector.

[0055] According to an aspect of the present application, performing data acquisition and basic calibration, comprising:

[0056] Performing response uniformization and black level correction using scene adaptive modeling based on low-rank plus sparse decomposition to avoid relying on mechanical shutter;

[0057] Detecting dynamic bad points through multi-scale consistency and temporal consistency;

[0058] Repairing dynamic bad points using structure tensor guided directional interpolation to protect edge structure and generating a basic calibration image sequence.

[0059] Beneficial effects, through the above technical solutions, the present application solves the problems of traditional index scale inconsistency, easy to be disturbed and poor curve shape, improves the scale consistency and anti-interference stability, optimizes the shape, and improves the convergence.BRIEF DESCRIPTION OF DRAWINGS BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a schematic diagram of an overall framework of a multi-dimension complementary sharpness evaluation method for infrared images.

[0061] Figure 2 is a schematic diagram of performing a physical gating fusion process.

[0062] Figure 3 is a schematic diagram of performing an online self-calibration process.

[0063] Figure 4 is a schematic diagram of constructing a physically consistent sharpness metric. DETAILED DESCRIPTION

[0064] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0065] Embodiment one, describes the overall framework of a multi-dimension complementary sharpness evaluation method for infrared images, as shown in Figure 1

[0066] By combining the physical characteristics of infrared imaging (such as heat diffusion, noise, gain) with mathematical modeling, a sharpness evaluation index with physical interpretability, cross-device comparability and unimodal searchability is constructed to solve the problems of poor sharpness curve shape, easy noise interference and scene change in traditional infrared focusing.

[0067] Step 101, obtaining an infrared image sequence, a device parameter set and environmental observation data.

[0068] In the present embodiment, the infrared image sequence is the original data stream to be evaluated for sharpness. The device parameter set is data representing the inherent properties of the imaging system, for example, it can include the bit depth of the sensor, the nominal focal length, the lens mechanism parameters, the sensor response characteristics and the noise equivalent temperature difference (NETD) and the like. The environmental observation data includes data representing the external conditions during imaging, for example, it can include scene temperature statistics, background temperature, humidity, visibility or optional weather data.

[0069] Step 102, performing basic calibration on the infrared image sequence to generate a basic calibration image sequence.

[0070] The base calibration is the prerequisite for physical quantity estimation and sharpness calculation. The main task of this step is to eliminate or suppress non-ideal factors in the original infrared image sequence, such as fixed pattern noise, bad pixels, and scale drift caused by automatic gain variation. Specifically, this step can include performing black level correction, response uniformization, dynamic bad pixel repair, and preliminary estimation of automatic gain. The generated base calibration image sequence has a more consistent dynamic range and a lower noise level, providing a robust data foundation.

[0071] In step 103, based on the base calibration image sequence, the device parameter set, and the environmental observation data, physical quantity estimation and normalization modeling are performed to obtain a set of normalized sharpness sub-indicators and a scene description vector.

[0072] This step extracts key physical and statistical features from the calibrated images and establishes a unified normalization framework. Physical quantity estimation can include: estimating the noise variance of the statistical flat area, estimating the accurate form of the automatic gain function, analyzing the structural tensor features and power spectrum slope, and estimating the scene equivalent temperature. Normalization modeling aims to build a closed mathematical model to eliminate the scale effects of noise, bit depth, automatic gain, and other factors on different sharpness indicators, making the indicators comparable in different devices and scenes. The output is a set of normalized sharpness sub-indicators. At the same time, the estimated physical features (such as anisotropy, temperature, spectral slope) are aggregated into a scene description vector to guide the subsequent fusion and enhancement.

[0073] In step 104, based on the set of normalized sharpness sub-indicators and the scene description vector, multi-dimensional complementary sharpness construction and unimodality enhancement are performed to output a unimodal enhanced sharpness curve and a real-time sharpness value.

[0074] This step constructs two or more types of complementary sharpness indicators in different dimensions in parallel. For example, a class of physically consistent sharpness indicators based on the thermal diffusion and optical defocus synthesis model, and a class of scale entropy sharpness indicators based on the thermal kernel multi-scale tensor decomposition. According to the scene description vector (especially the scene equivalent temperature), a physical gating mechanism based on the Boltzmann distribution is used to adaptively fuse the two types of indicators to obtain a fused sharpness initial value. A unimodal enhancement transformation (such as peak lifting and sidelobe suppression) is applied to the fused value, and a unimodality index is calculated online. The transformation parameters are adaptively adjusted by a short window micro-scan to maximize the unimodality index, and finally a unimodal enhanced sharpness curve with good shape and easy to search is output, as well as the real-time sharpness value of the current focus.

[0075] In this embodiment, the method can further include performing a focus search and control loop. This step reads the unimodal enhanced sharpness curve and real-time sharpness value produced in step 104 as the target plane for the focus search. The search strategy can employ, for example, interval bracketing, three-point parabolic fitting to estimate the peak position and local curvature, and adaptively adjust the search step size according to the curvature. When the risk of sidelobes or multi-peaks is detected, this step can be linked back to dynamically adjust the sidelobe suppression parameter of the unimodal enhancement transform in step 104, actively reshape the sharpness curve profile to facilitate the search, and form a closed loop of measurement and control.

[0076] In this embodiment, the method can further include performing statistical evaluation and model self-calibration. This step is performed in the background or offline state, aggregates the records of the focus search process (such as the number of convergence steps, the number of sidelobe events), and evaluates the long-term performance of the overall focus quality and unimodal indicators. Based on the statistical evaluation results, the model parameters used in steps 103 and 104 can be self-learned and updated, such as updating the free energy metric parameters of the physical gating fusion (such as Boltzmann fusion) in step 104, or updating the parameter boundaries of the unimodal enhancement transform, which constitutes the highest level of adaptive closed loop of the system, and improves the robustness of the method to long-term working condition changes.

[0077] Embodiment Two, describes the execution of physical parameter estimation and normalization modeling in step 103 of Embodiment One. Specifically as follows:

[0078] Step 201, perform physical parameter estimation. To extract the physical and statistical characteristics required for subsequent normalization, fusion and enhancement from the basic calibration image sequence.

[0079] Specifically, this step can include:

[0080] Estimate noise variance: for example, by a robust criterion combining block-level variance and median absolute deviation, identify flat areas in the image, and after removing edges and textures, statistically analyze the pixel fluctuations in the flat areas to obtain a robust estimate of the sensor noise variance.

[0081] Estimate bit depth and dynamic range: read the bit depth information (such as 12 bits or 14 bits) in the device parameter set, and establish a bit depth normalization coefficient across devices, such as linear mapping to normalize the grayscale to a unified dynamic range (such as 0-65535).

[0082] Estimate structure and spectral features: for example, calculate the structure tensor for each frame of image and solve the eigenvalues to obtain the anisotropy strength and principal direction; at the same time, perform logarithmic linear fitting on the radial average power spectrum to obtain the power spectrum slope. These features reflect the scene content (such as sparse edges or dense textures).

[0083] Estimate scene equivalent temperature: For example, using the brightness temperature coarse mapping (as in Example 6) or the device calibration relation obtained in the basic calibration step, estimate the temperature distribution of the current frame, and calculate the peak value, peak width and overall temperature level to obtain the scene equivalent temperature.

[0084] Step 202, generate scene description vector. Aggregate the multiple physical parameters estimated in step 201 to form a structured data vector, i.e. the scene description vector.

[0085] Preferably, the scene description vector can aggregate but is not limited to anisotropy intensity, principal direction, power spectrum slope, scene equivalent temperature and temperature distribution characteristics. This vector constitutes a compact description of the physical characteristics of the current imaging scene, which will serve as the basis for subsequent steps (such as Example 4) to perform physical gating fusion and single-peak enhancement parameter prior setting.

[0086] Step 203, estimate automatic gain function and gain energy factor. Automatic gain control (AGC) is one of the main sources of scale drift of the clarity index in infrared imaging. In order to achieve normalization, its form must be accurately estimated.

[0087] Preferably, this step combines the preliminary estimation results in the basic calibration (as in Example 6).

[0088] In the basic calibration phase, local fitting of the gray scale mapping can be used to identify the segments and switching points of the automatic gain. On this basis, this step performs more refined fitting for each segment identified, for example, using a monotonic constrained spline to fit the gray scale histogram within the segment to construct the automatic gain function.

[0089] Further, the derivative of the automatic gain function and the weighted norm of the current gray scale distribution are calculated to obtain the gain energy factor G, which quantitatively describes the amplification (or compression) effect of the automatic gain on the overall contrast of the image.

[0090] Step 204, construct closed-form normalization mapping. To eliminate the scale effects of estimated noise, bit depth and automatic gain on clarity sub-indices.

[0091] In this embodiment, for each type of clarity sub-index S i to be calculated subsequently (such as the physically consistent clarity S phys in Example 3 or the thermal kernel multi-scale tensor clarity S tensor ), a corresponding normalization mapping is established. Preferably, the closed-form normalization mapping (also known as NAA-Norm-IR) uses the following unified normalization formula: Stilde i =(S i -μ i (σn 2 )) / σ i (σ n 2 )*2 -b *G -1 ; wherein Stilde i is the normalized articulation sub-index; S i is the original calculated articulation sub-index; σ n 2 is the noise variance; b is the bit depth; G is the gain energy factor. μ i (σ n 2 ) and σ i (σ n 2 ) are the theoretical expectation and theoretical standard deviation of the articulation sub-index S i under the condition of only noise variance σ n 2 .

[0092] In a preferred implementation, in order to determine μ i and σ i , the corresponding index noise response model needs to be selected according to the mathematical definition of S i (for example, is it a frequency domain product class or a tensor decomposition class). The specific values of μ i and σ i can be obtained by analytical calculation or robust correction by sample quantile statistics according to the model in the flat region. Through the formula, Stilde i calculated under different devices, different gains, and different noise levels are unified to the same scale, forming a set of normalized articulation sub-indices, which provides the same scale comparable input for subsequent physical gating fusion (such as embodiment four).

[0093] Step 205, all outputs of this embodiment are summarized to form a set as the input of the subsequent steps. The set preferably includes: a set of normalized transformers instantiated for each type of sub-index (i.e. the formula of step 204 and its parameters), the scene description vector generated in step 202, and the noise variance estimated in step 201, etc.

[0094] Embodiment three, describes the specific implementation of the multi-dimensional complementary articulation construction in step 104 of embodiment one. In this embodiment, two types of core articulation indices with physical meaning and structural insight are constructed in parallel, which together constitute the input source of the normalized articulation sub-index set mentioned in embodiment two.

[0095] Step 301, construct a physically consistent articulation index, as shown in Figure 4 .

[0096] This step aims to construct a sharpness index S based on an imaging physics model. phys This model unifies optical defocusing blur with the thermal diffusion blur effect unique to infrared.

[0097] Specifically, this embodiment introduces a synthetic fuzzy variance model, the expression of which can be: σ eff 2 =σ opt 2 +2*κ*t. Where, σ eff 2 It is the total equivalent fuzzy variance; σ opt 2 The blur variance caused by lens optical defocus is a function of the focal length z, and it reaches its minimum at the optimal focus; κ is the thermal diffusivity; and t is the thermal diffusivity equivalent time. This model attributes the blurring in infrared imaging to the combined effect of the optical system and thermal conduction (or their mathematical equivalents).

[0098] Based on this model, the physical consistency sharpness index S phys It is constructed as a frequency-domain weighted integral:

[0099] S phys =∫(W(f)*|I hat (f)| 2 df), the integration interval is from f=0 to infinity. Where, I hat (f) is the amplitude spectrum of the basic calibration image sequence; f is the spatial frequency.

[0100] The frequency domain weighting function W(f) is designed to have a form related to the synthesized modulation transfer function (MTF). Preferably, an inverse weighting design is used, such as W(f) = f 2 *exp(2*π 2 *σ ref 2 *f 2 ), where σ ref It is the nominally best imaging (i.e., σ) opt The fuzzy variance corresponding to the minimum value.

[0101] W(f) assigns higher weights to high-frequency components that attenuate due to ambiguity, in σ opt 2 When deviating from the optimal point (i.e., when the blur increases), S phys The value will decrease monotonically, ensuring the unimodality of this indicator.

[0102] In a preferred implementation, to improve S phys Robustness of computation, in calculating amplitude spectrum I hat (f) Prior to this, a preprocessing step may be performed on the basic calibration image.

[0103] For example, to reduce the spectral leakage, a two-dimensional cosine window with boundary mirror extension can be employed. In addition, to avoid the energy of low-frequency background (e.g. large area of sky or wall) dominating the integration result, a flat region mask can be used to suppress or down-weight the energy corresponding to very low frequencies and the direction of stripe noise in the frequency domain. The calculated raw S phys values are sent to the normalization transformer constructed in Example Two (step 204) to output normalized physical-consistent sharpness S phys tilde .

[0104] Step 302, construct a thermal kernel multi-scale tensor sharpness indicator.

[0105] This step aims to construct a complementary sharpness indicator S tensor that provides different dimensional information from S phys based on the scale-space structure and energy concentration of the image.

[0106] Specifically, this step first needs to construct a three-dimensional tensor T(x, y, s). Unlike the traditional Gaussian pyramid, the scale axis s in this embodiment is driven by the physical thermal kernel (i.e. the Green's function of the heat conduction equation). For example, a discrete thermal kernel scale set s k = sqrt(2 * kappa * t k ), where kappa is the thermal diffusivity and t k is a set of log-uniformly distributed equivalent times.

[0107] Convolve the base calibration image sequence with the thermal kernel scale set s k to generate a set of scale images, and stack these scale images along the s axis to form the three-dimensional tensor T(x, y, s).

[0108] On this basis, perform tensor decomposition on the three-dimensional tensor T(x, y, s), such as high-order singular value decomposition (HOSVD) or Tucker decomposition. The decomposition operation decomposes the tensor into a core tensor and factor matrices of each mode (space, scale).

[0109] This step extracts key features from the decomposition results to construct the sharpness indicator. For example, extract the first two singular values λ1, λ2 representing the directional energy, i.e. the directional energy singular values λ1, λ2, and the factor matrix U3 representing the scale mode.

[0110] The thermal kernel multi-scale tensor sharpness indicator S tensor can be composed of the following formula:

[0111] S tensor= (λ1 / λ2) * exp(-H(U3)); where the (λ1 / λ2) term represents the directional energy concentration. When the image is sharp, the energy is concentrated in a few dominant directions, making λ1 much larger than λ2. When the image is blurred, the energy is uniformly distributed in all directions, and the ratio approaches 1.

[0112] H(U3) is the information entropy of the scale matrix U3. The exp(-H(U3)) term accordingly represents the scale energy concentration. When the image is sharp, its energy is concentrated in a few key scales, H(U3) is small, and the term is large. When the image is blurred, the energy is spread to all scales, H(U3) is large, and the term is small.

[0113] In summary, S tensor is a multiplicative measure, taking its maximum value when the energy of the image exhibits strong concentration in both direction and scale (i.e., the image is sharp).

[0114] The original S tensor value computed is also fed into the normalized transformer constructed in Example Two (step 204) to produce the normalized hot-nuclear multi-scale tensor sharpness Stilde tensor . Meanwhile, the tensor decomposition process also outputs the tensor decomposition auxiliary statistics (e.g., the scale energy spectrum) for the sidelobe risk assessment in Example Four.

[0115] According to one aspect of the present application, in Example Three, the physical model parameters κ and the thermal diffusion equivalent time t involved in steps 301 (constructing the physically consistent sharpness indicator) and 302 (constructing the hot-nuclear multi-scale tensor sharpness indicator) are further explained as follows:

[0116] The synthetic blur variance σ eff 2 = σ opt 2 + 2 / κt in step 301 and the hot-nuclear scale s = sqrt{2 / κt} in step 302 are a kind of mathematical modeling based on physical heuristics. Specifically, this model borrows the mathematical form of the Green's kernel of the heat conduction equation, using the 2 / κt term to parameterize the unique, non-optical blur component in the infrared image (this blur characteristic is similar in mathematical form to thermal diffusion). This is a mathematical analogy, not a direct measurement of the physical thermal diffusion process.

[0117] In this model, κ is a model coefficient, whose value is not the physical thermal diffusion rate, but is preferably a system-level calibration parameter. For example, κ can be obtained as an empirical constant representing the rate of evolution of the blur characteristics of the infrared system with the equivalent time t, by model fitting on a set of infrared images with known different blur levels.

[0118] Similarly, t (thermal diffusion equivalent time) is also a model parameter for characterizing the analysis scale. Its value (or t k discrete set) can be set heuristically a priori according to the typical size of the target to be analyzed. For example, to analyze the blur characteristics of fine textures or small targets (e.g. 3x3 pixels) in an image, a set of smaller t values can be set, such as t set ={0.1, 0.2, 0.5} (units can be arbitrary time units, just as model parameters). Correspondingly, to analyze the structure or blur characteristics of medium-sized targets (e.g. 20x20 pixels), a set of t values in the medium range can be set, such as t set ={0.8, 1.5, 3.0}, so that the model can match the blur evolution characteristics at different scales.

[0119] According to one aspect of the present application, the construction of physically consistent sharpness indicators and scale alignment can also be:

[0120] Read the basic calibration image sequence and the flat area mask, select a two-dimensional cosine window and boundary mirror extension for each frame to reduce spectral leakage and edge effects, calculate the amplitude spectrum and frequency sampling specification, and map the flat area mask to the frequency domain for subsequent band suppression, to obtain the output amplitude spectrum and frequency sampling specification.

[0121] Based on the amplitude spectrum, frequency sampling specification and flat area mask, construct a spectral suppression mask with the low frequency energy of the flat area as a reference, down-regulate the weight of the very low frequency and the direction of the stripe noise to avoid background energy dominating the integration, and obtain the spectral suppression mask.

[0122] According to the nominal best imaging variance in the scene description vector and the device parameter set, set the weight function according to the synthesis of thermal diffusion and optical defocus. According to the power spectrum slope and anisotropy intensity in the scene description vector, fine-tune the weight slope and cutoff buffer band at the high frequency end to obtain the weight function.

[0123] According to the amplitude spectrum, spectral suppression mask and weight function, perform numerical integration in the frequency domain, combine the spectral suppression mask for low frequency blanking and stripe suppression, and apply anti-aliasing protection to the high frequency tail (automatically converge to the upper limit according to the frequency sampling specification). Obtain the physically consistent sharpness.

[0124] Read the physically consistent sharpness and the set of normalization transformers, call the closed-form normalization mapping of NAA-Norm-IR, and obtain the normalized physically consistent sharpness.

[0125] Embodiment four, which describes the fusion, enhancement and self-calibration process in the multi-dimensional complementary sharpness construction and unimodality enhancement of step 104 in embodiment one. This embodiment is based on the normalized sharpness sub-indicator set (e.g. Stilde physand Stilde tensor ) and the scene description vector output by Example Two as inputs.

[0126] Specifically, the method comprises the following steps:

[0127] Step 401, physical gating fusion is performed on the two types of indicators. That is, according to the scene characteristics and the real-time reliability of each indicator, Stilde phys and Stilde tensor are adaptively fused into a single fused definition S fuse . As shown in FIG. 4. Figure 2

[0128] In this embodiment, the determination of the fusion weight is based on a virtual free energy metric E i . For the i-th indicator in Stilde phys and Stilde tensor , its free energy E i is intended to quantify its unreliability or the tendency to easily lead to search failure.

[0129] The virtual free energy metric E i can be evaluated by applying a virtual defocus perturbation and noise injection to the current image. Specifically:

[0130] By applying a Gaussian smoothing with a very small standard deviation (simulating micro-defocus) to the image, the descending rate and curvature of the indicator Stilde i are observed to obtain a unimodality prior score M i .

[0131] By injecting noise consistent with the noise variance estimated in Example Two in the flat region, the variance of Stilde i is counted to obtain a noise sensitivity score Noise i .

[0132] By analyzing the tensor decomposition auxiliary statistics (such as the scale energy spectrum) output by Example Three, the multi-peak risk is estimated to obtain a sidelobe tendency score Sidelobe i .

[0133] Preferably, the free energy metric E i can be constructed as a linear combination of the above scores:

[0134] E i =a1*(1-M i )+a2*Noise i +a3*Sidelobe i ; where a1, a2, a3 are calibration coefficients, which can be updated by offline self-learning. The virtual free energy metric E i ​The higher, the worse the unimodality of the indicator Stilde i , the more noise-fearing or the more prone to side-lobes.

[0135] On this basis, the free energy measure E i and the scene equivalent temperature T eff of the scene description vector extraction in Example Two are used to calculate the physical gating weight w i through the Boltzmann distribution:

[0136] w i =exp(-E i / (k*T eff )) / (∑ j (exp(-E j / (k*T eff ))));where k is the Boltzmann constant (or set to 1). The physical meaning of this physical gating is: the lower the free energy E i (the more reliable the indicator), the higher the weight w i obtained; and the higher the scene temperature T eff , the smoother the weight distribution (softmax effect).

[0137] Finally, the fusion clarity S fuse is obtained by weighted summation: S fuse =∑ i (w i *Stilde i ).

[0138] Step 402, apply unimodal enhancement transformation to the fusion clarity and perform online self-calibration.

[0139] The goal is to reshape the fusion clarity S fuse (may still have peak flat or shoulder side-lobes) into the final clarity S star with sharp unimodal, easy to search.

[0140] First, apply unimodal enhancement transformation to the fusion clarity S fuse . The transformation preferably takes the following form:

[0141] S star =log(1+exp(α*S fuse ))-γ*LSE k (S fuse *d k ); where the first term log(1+exp(...)) is a Softplus function, and the parameter α is used to control the lifting and sharpening of the peak. The second term is a side-lobes suppression term, γ is the suppression intensity parameter, and LSE k is a log-sum-exponential smoothing max operator, dk is a local kernel.

[0142] In a preferred implementation, the initial value or range of values of α and γ, as well as the shape of kernel d k , are not fixed but are set a priori by the scenario description vector (SDV) of embodiment two. For example, if the anisotropy strength in the SDV is high, the initial range of γ can be raised, and kernel d k can be shaped to match the anisotropy direction.

[0143] An online self-calibration is performed to maximize the unimodality of the sharpness curve, as shown in Figure 3 .

[0144] The core of this process is to define a computable unimodality index M. This M must quantify both the peak sharpness and the side lobe suppression of the curve.

[0145] In this embodiment, to compute the unimodality index M, the system performs a short window microscan around the current focus, for example, a small step focus micro-motion of plus or minus 2 to 3 steps with very small step size around the current focus z, and collects the corresponding local S star sampling sequence.

[0146] Based on this local sampling sequence, the unimodality index M is computed, which can be defined as:

[0147] M = min z (d 2 / dz 2 [smooth(log(S star (z)))]) - η * (secondary peak height / primary peak height);

[0148] where d 2 / dz 2 [smooth(log(S star (z)))] is the second derivative (i.e., the curvature) of the log sharpness curve, min(...) means taking its minimum value (representing the peak sharpness, the more negative the better); the second term is the ratio of the secondary peak height to the primary peak height (a side lobe penalty term); η is a weighting coefficient.

[0149] To maximize this unimodality index M (i.e., to make M as small as possible, or some positive transformation of it), the parameters of the unimodality enhancement transform (α, γ) or the weights of the physical gating fusion (w i ) are updated online.

[0150] In a preferred implementation, this update can use optimization algorithms such as limited amplitude mirror gradient steps to make small updates to α and γ. If the secondary peak ratio is detected to be too high, this step will also call kernel d kadaptive adjustment to target the sidelobes.

[0151] After 1-2 rounds of iterative update, the system generates the final unimodal enhanced definition curve (i.e. S star (z) as a function of the real-time definition value (i.e. S star value corresponding to the current focal point z).

[0152] According to one aspect of the present application, in embodiment four, the scene equivalent temperature T eff and its application in the Boltzmann distribution are further described as follows:

[0153] In this step, the Boltzmann distribution is used to calculate the physical gating weight w i is a mathematical borrowing. Its function is equivalent to the Softmax weight distributor commonly used in machine learning, which is used to soft-weight two (or more) definition sub-indicators (S phys and S tensor ). E i is a virtual free energy or cost function composed of measurable indicators (such as unimodality and noise sensitivity).

[0154] In this model, T eff is not an accurate measurement of the physical temperature, but a temperature parameter of this Softmax distributor, which is used to control the sharpness or smoothness of weight distribution. Specifically, when a lower T eff value is set, the weight distribution tends to be sharp (i.e. winner takes all), so that the indicator with significantly lower free energy E i obtains most of the weight (e.g. w1=0.9, w2=0.1). When a higher T eff value is set, the weight distribution tends to be smooth (e.g. w1=0.55, w2=0.45), which is suitable for the case where the two indicators E i are close or the scene is uncertain.

[0155] The specific value of T eff can be adaptively set according to the scene temperature distribution characteristics (such as peak width and peak number) estimated in embodiment two (step 201). This is a heuristic mapping. For example, if the temperature distribution statistics show a single, sharp peak (indicating a simple scene with concentrated contrast), the system can determine that the scene is highly certain and set a lower T eff value (e.g. T eff normalized parameter value is 0.5) for decisive weight distribution. Conversely, if the temperature distribution shows multiple peaks or a broad flat top (indicating a complex scene with target and background temperatures intertwined), the system can determine that the scene is highly uncertain and set a higher Teff value (e.g., T eff The normalized parameter value is 2.0), to perform more conservative, more smooth weight fusion.

[0156] According to one aspect of the present application, the free energy metric construction and the Boltzmann gate fusion can also be:

[0157] According to the basis calibration image sequence, the normalized physical consistent sharpness and the normalized thermal kernel multi-scale tensor sharpness, the current frame is constructed with a Gaussian smoothing of a minimum standard deviation to simulate a slight increase in synthetic blur variance, and the two types of sharpness are recalculated respectively to obtain a relative amplitude and a local second-order curvature, forming a unimodal prior score (one for each type of index). The unimodal prior score is output.

[0158] Based on the basis calibration image sequence and noise variance, Gaussian noise consistent with noise variance is injected in the flat area, and the two types of sharpness are repeatedly calculated to obtain noise sensitivity scores (one for each type of index).

[0159] Read the tensor decomposition auxiliary statistics and the normalized thermal kernel multi-scale tensor sharpness, extract the secondary peak and primary peak energy ratio from the scale energy spectrum, and quantify the potential multi-peak or scale sidelobe to obtain a sidelobe tendency score.

[0160] Based on the unimodal prior score, the noise sensitivity score and the sidelobe tendency score, the free energy metric of each index is constructed by linear combination to obtain a free energy metric set.

[0161] Read the free energy metric set and the scene equivalent temperature, and use the Boltzmann distribution to calculate the gate weight to obtain a physical gate weight.

[0162] According to the normalized physical consistent sharpness, the normalized thermal kernel multi-scale tensor sharpness and the physical gate weight, the fusion sharpness is calculated, the stability of the time series mutation is checked (if the weight mutation exceeds the threshold, the smoothing replacement is triggered), and the fusion sharpness initial value is obtained.

[0163] According to one aspect of the present application, the unimodal enhancement transform and the sidelobe suppression can also be:

[0164] Read the physical gate weight, the anisotropy strength in the scene description vector and the power spectrum slope, and consider strong anisotropy and steep high frequency spectrum as a priori of sidelobe risk, set the range of the unimodal enhancement parameter initial value and the sidelobe suppression parameter initial value

[0165] According to the fusion sharpness initial value, the unimodal enhancement parameter initial value and the sidelobe suppression parameter initial value, perform:

[0166] S star =log(1+exp(α•S fuse))−γ•LSE k (S fuse ∗d k ), where the unimodal enhancement parameter a raises the peak top, the sidelobe suppression parameter g weakens the sidelobe, and the convolution kernel d k The scale and shape of d fuse are guided by the anisotropy strength. Considering numerical stability, a soft saturation protection is performed on d

[0167] According to the scene description vector and the physical gating weight, the direction weight and the scale weight of d k are fine-tuned, so that the suppression is stronger in the high-risk direction and is not too suppressed in the main information direction, and the convolution kernel weight adjustment quantity is generated.

[0168] The unimodal enhancement clarity initial value is read, and the boundary points of the curve are protected to avoid edge distortion caused by convolution from causing misjudgment of short window micro scanning, and the unimodal enhancement clarity initial value after boundary protection is output. The unimodal enhancement clarity initial value after boundary protection is output.

[0169] Embodiment five, this embodiment details the implementation of the focus search and control closed loop in embodiment one. This embodiment takes the unimodal enhancement clarity curve (target surface) and the real-time clarity value (current value) output by the above-mentioned embodiment four as input.

[0170] Step 501, focus search is performed.

[0171] In a preferred embodiment, the search does not start from blind global scanning.

[0172] Preferably, the first step of the search is initial interval bracketing. This step uses historical priori (such as the best focus position of the last time focusing and local curvature, which can be provided by the statistical module of embodiment eight) and combines the local difference (first derivative estimation) of the current focus to intelligently construct an initial search interval containing the main peak. For example, if the scene has not changed significantly (which can be determined by histogram or S.D.V. change), the search interval can be scaled and centered on the peak position of the last time, reducing the invalid search range.

[0173] Step 502, three-point fitting and curvature adaptive step are performed.

[0174] Within the bracketing interval determined in step 501, the system performs the search.

[0175] Specifically, the system collects at least three focus positions z1, z2, z3 corresponding to real-time clarity values S star (z1), S star (z2), and S star (z3) in the interval.

[0176] In a preferred implementation, the selection of sampling points is constrained. The system will actively avoid the automatic gain switching adjacent bands identified in embodiment two (or embodiment six). This is because the switching of automatic gain will cause the sharpness value S star A mutation unrelated to focusing occurs, and if the sampling point falls into this region, it will cause the subsequent fitting to produce a false peak.

[0177] After obtaining the sampling points, the system fits a parabola to the three sets of data (z i , S star (z i )) to obtain the candidate focal point position (i.e. the vertex of the parabola) and the local curvature estimate (i.e. the second-order coefficient of the parabola).

[0178] The local curvature estimate is used to implement a curvature-adaptive step size. Specifically, according to the local curvature estimate, the step size for the next movement is set. For example, if the curvature estimate value is large (the peak top is very sharp), the step size for the next movement should be correspondingly reduced for fine search; if the curvature estimate value is small (the peak top is very flat), the step size can be appropriately increased for rapid approximation.

[0179] Step 503, perform sidelobe detection and reverse linkage. During the search process, the system continuously detects the sidelobe risk.

[0180] The basis for determining the sidelobe risk can include: during the search process, according to the ratio of the secondary peak to the main peak (of the collected S star sequence), or the curvature inconsistency (such as the fitting residual being too large, or the curvature sign being abnormal) (of the parabola fitting), to determine.

[0181] In a preferred implementation, as in step 502, if the candidate focal point position is adjacent to the automatic gain switching band, it will also be determined as a sidelobe risk (false peak risk).

[0182] In response to the sidelobe risk, the system generates a sidelobe suppression parameter increment.

[0183] The increment is written back to the multi-dimension complementary sharpness construction and unimodality enhancement step (i.e. embodiment four).

[0184] The specific write-back operation can include:

[0185] Increasing the value of the sidelobe suppression parameter γ of the unimodal enhancement transform in embodiment four.

[0186] Preferably, adjusting the weights of the convolution kernel d k in embodiment four to impose stronger suppression in the specific scale or direction detected.

[0187] Optionally, temporarily increasing the scene equivalent temperature T effThe lower bound of the range is used to smooth the fusion weights, reducing the volatility of the curve.

[0188] Through this reverse linkage, the sharpness curve S star (z) is dynamically changed (becomes smoother or has lower sidelobes) in the calculation of the next frame, to proactively adapt the search algorithm to make it easier to find the true peak.

[0189] Step 504, generate the focus drive sequence.

[0190] This step converts the candidate focus position or step suggestion produced in step 502 into precise motor control instructions.

[0191] Obtain the lens mechanism model in the set of device parameters. This model preferably includes the inverse gap, static and dynamic friction models of the lens, as well as maximum speed, acceleration and jerk constraints.

[0192] Based on the candidate focus position or step suggestion, generate an initial drive sequence using a jerk-constrained trajectory planning (e.g. S-curve trajectory planning). This ensures smooth motor start-stop and avoids inaccurate sharpness measurement due to vibration.

[0193] In combination with the lens mechanism model, perform inverse gap compensation (e.g. additional compensation displacement when the direction of motion changes) and feedforward friction compensation (e.g. pre-apply a torque to counter friction according to the speed curve) on the initial drive sequence to generate the final focus drive sequence.

[0194] Step 505, determine the optimal focus position.

[0195] The search process needs a robust termination criterion.

[0196] In a preferred implementation, the termination criterion is not based on a single threshold of the sharpness curve S star but uses a multi-criteria termination criterion.

[0197] This criterion can combine the following indicators:

[0198] Position convergence: for example, the change in the candidate focus position obtained by three-point fitting twice in a row is less than the minimum step distance of the motor.

[0199] Curvature convergence: the local curvature estimate has stabilized within a reasonable range (e.g. greater than a certain threshold, indicating that it is at the peak; and the rate of change is small enough).

[0200] Uncertainty evaluation: the focus position uncertainty estimated based on curvature and noise level has converged to an acceptable range.

[0201] When the multi-criteria termination criterion is satisfied, the search is stopped and the last candidate focus position is output as the optimal focus position.

[0202] Embodiment six, a preferred and detailed implementation of the base calibration in step 102 of embodiment one. This embodiment focuses on providing a high-quality, high-stability base calibration image sequence for all subsequent steps by correcting and anchoring the early stage of data acquisition.

[0203] Step 601, shutter-independent response homogenization and black level correction is performed.

[0204] This step supports the definition of response homogenization and black level correction. Traditional infrared correction, i.e. non-uniformity correction (NUC), relies heavily on the mechanical shutter (or called shutter) in front of the lens to close to obtain a uniform radiation field for calibrating the pixel response. However, the action of the mechanical shutter inevitably introduces slight vibration and can cause the thermal balance of the sensor focal plane to be broken instantaneously. In the process of performing precise focusing (such as embodiment five), the disturbance caused by the out-of-focus factor will cause the sharpness curve to produce abnormal jumps, seriously interfering with the convergence of the search algorithm.

[0205] To solve this problem, in this embodiment, a method based on scene adaptive modeling is preferably used to achieve shutter-independent correction. For example, a low-rank plus sparse decomposition (LRSD) model can be used. This model assumes that within a short time window, the infrared image sequence can be modeled as the superposition of three parts: a real scene that changes slowly over time (corresponding to the low-rank part in mathematics), a fixed pattern noise in the spatial domain (such as stripe noise, corresponding to the sparse part in mathematics), and a random noise term.

[0206] Specifically, by performing robust principal component analysis (RPCA) or similar low-rank matrix recovery algorithm on the image sequence (e.g. 100 frames) within a time window, the fixed pattern noise can be robustly separated from the sparse part, and the black level (fixed bias) and gain coefficient of each pixel (i.e. non-uniformity correction coefficient) can be estimated in a robust regression manner. This method does not rely on the mechanical shutter and can continuously and adaptively update the correction coefficients during normal imaging of the system, avoiding the interference of vibration and thermal disturbance while ensuring the real-time and accuracy of the correction.

[0207] Step 602, dynamic bad pixel self-updating and structure preserving interpolation is performed. This step supports the definition of dynamic bad pixels and directional interpolation. Traditional bad pixel correction relies on a static bad pixel table calibrated at factory. However, after long-term operation or experiencing severe temperature changes, new bad pixels or intermittent bad pixels may appear on the sensor, which cannot be covered by the static bad pixel table. These bad pixels usually appear as constant bright or dark spots in the image, and if not handled, they will be mistaken for very strong high-frequency details (edges) by subsequent sharpness algorithms (such as embodiment three), resulting in a fixed false peak on the sharpness curve that is unrelated to the true focus.

[0208] In this embodiment, a dynamic bad pixel self-updating strategy is adopted. For example, by multi-scale consistency test (compare the statistical characteristics of a pixel and its neighborhood at different scales) and time consistency test (detect whether the gray value of a pixel jumps non-physically in time domain), new bad pixels and intermittent bad pixels are detected and located in real time, and a dynamically updated bad pixel mask is generated.

[0209] When repairing the detected bad pixels, to protect the true edge structure of the image, a structure tensor guided directional interpolation is preferably used. Specifically, traditional mean or median interpolation is isotropic, and when the bad pixel falls on an edge, this interpolation will pull the average gray value of the two sides of the edge, resulting in edge blurring. In this embodiment, the structure tensor of the bad pixel neighborhood (e.g. a 5x5 window) is first calculated, and its eigenvectors are solved to determine the main direction of the neighborhood (i.e. the direction of the edge). The interpolation process (e.g. one-dimensional linear or spline interpolation) will be carried out along the main direction (i.e. parallel to the edge direction), rather than perpendicular to the edge. Directional interpolation can avoid introducing additional blurring when repairing edge bad pixels, and maximizes the protection of high-frequency information required for subsequent sharpness calculation.

[0210] Step 603, environmental assimilation and bright temperature coarse mapping anchoring is performed. This step is used to preliminarily associate the sensor output gray value (e.g. 0-4095) without physical units with the bright temperature with clear physical meaning. It provides a physical basis for estimating the scene equivalent temperature (T eff ) key physical parameter in embodiment two.

[0211] Specifically, this step utilizes known sensor response curves (if any) and factory calibration constants from the device parameter set, combined with environmental observation data (e.g., the average background temperature of the scene or sky temperature measured by external sensors) as one or more radiation anchor points. For example, it can be assumed that the true brightness temperature of a large background area in the image (such as the sky or distant mountains) should be close to the background temperature value in the environmental observation data. Using this anchor point (i.e., a set of [grayscale value, brightness temperature value] corresponding points), the system can construct a simplified (e.g., piecewise linear or polynomial) coarse mapping from grayscale to brightness temperature. This step incorporates environmental physical information (rather than pure image statistics) into the early calibration, reducing the need for subsequent T... eff The deviation in the estimate.

[0212] Step 604: Perform coarse estimation and segmentation of the automatic gain function. This step is a prerequisite for the precise estimation of the automatic gain function (step 203) in Example 2.

[0213] Automatic gain control (AGC) in infrared imaging systems is typically piecewise linear (or nonlinear), with different gain slopes (i.e. contrast stretching) in different grayscale input ranges.

[0214] In this embodiment, monotonic constraint splines are preferably used to locally fit the frame-by-frame grayscale mapping (i.e., the relationship between input and output grayscale), and combined with the detection of grayscale change rate and saturation pixel ratio, to explicitly identify the segmentation points and switching points of automatic gain. The output gain segment division information has a dual purpose: first, it is used in Embodiment 2 (step 203) to perform individual refined fitting on each segment, improving the estimation accuracy of the gain energy factor G; second, it is used in Embodiment 5 (steps 502, 503) to treat these switching bands as high-risk areas and actively avoid them during focus search, so as to prevent the focus algorithm from misjudging them as false peaks due to sudden gain changes.

[0215] Step 605: Generate data quality tags and consistency reports.

[0216] This step preferably involves real-time evaluation of the quality of the generated base calibration image sequence. For example, its signal-to-noise ratio (which can be estimated using the flat area and noise variance from Example 2), the proportion of saturated pixels (overexposed or underexposed), and scene stability (e.g., through inter-frame difference estimation) can be calculated.

[0217] The evaluation results are used to form a quality label. This label is used as a confidence prior in subsequent steps to improve the robustness of the algorithm. For example, if the quality label indicates that the current signal-to-noise ratio is low, Embodiment 2 (step 201) can automatically increase the threshold for identifying flat regions when estimating the noise variance; and Embodiment 4 (step 401) can automatically increase the threshold for calculating the free energy E. i When this is the case, a noise sensitivity score can be given. iHigher weights (i.e., increasing the corresponding coefficient a2) make the fusion result more inclined towards the sub-index with stronger noise resistance.

[0218] Example 7: Calculation process of physical gating fusion and unimodality index. This example illustrates the calculation process of free energy metric, Boltzmann gating weight, and unimodality index.

[0219] Step 701, calculation of physical gating fusion.

[0220] This step exemplifies the computation flow of step 401 (physical gating fusion) in embodiment four.

[0221] Suppose that at a certain moment, the system has completed the calculations of Example 3 and obtained two normalized sharpness sub-indices (i.e., a set of normalized sharpness sub-indices):

[0222] Physical consistency sharpness Stilde phys =1.50; Thermonuclear Multiscale Tensor Resolve Stilde tensor =1.20;

[0223] Meanwhile, the system has completed the calculations for Example 2, obtaining the scene equivalent temperature T from the scene description vector. eff To simplify the calculation, we assume k*T. eff After calibration, the equivalent normalized energy of the product of Boltzmann constant and temperature is 0.8.

[0224] The system executes the evaluation process of Example 4 (step 401), and obtains the free energy measure E of each of the two indicators through virtual defocus perturbation, noise injection, and sidelobe risk assessment. i Assume the evaluation results are as follows:

[0225] For Stilde phys Its unimodal prior score M i =0.9 (Good), Noise Sensitivity i =0.2 (low), sidelobe tendency i =0.1 (low).

[0226] For Stilde tensor Its unimodal prior score M i =0.7 (Medium), Noise Sensitivity i =0.6 (slightly high), sidelobe tendency i =0.5 (Medium).

[0227] Assume the free energy formula defined in Example 4 is E i =a1*(1-M i)+ a2Noise i + a3Sidelobe i , and the calibrated coefficients (which can be updated by embodiment eight) are currently all a1=1.0, a2=1.0, a3=1.0.

[0228] Then E phys =1.0*(1-0.9)+1.00.2+1.00.1=0.4. Then E tensor =1.0*(1-0.7)+1.00.6+1.00.5=1.4.

[0229] E phys is much smaller than E tensor , indicating that the system is more reliable in determining Stilde phys in the current frame.

[0230] On this basis, the system calculates the physical gating weight w i : w phys =exp(-E phys / (kT eff )) / (exp(-E phys / (kT eff ))+exp(-E tensor / (k*T eff ))) by the Boltzmann distribution.

[0231] w phys =exp(-0.4 / 0.8) / (exp(-0.4 / 0.8)+exp(-1.4 / 0.8));

[0232] w phys =exp(-0.5) / (exp(-0.5)+exp(-1.75));

[0233] w phys =0.6065 / (0.6065+0.1738)=0.777 (i.e., 77.7%).

[0234] Correspondingly, w tensor =1-w phys =0.223 (i.e., 22.3%).

[0235] Finally, the fused sharpness S fuse is obtained by weighted summation:

[0236] S fuse =w phys *Stilde phys +w tensor *Stilde tensor S fuse=0.777*1.50+0.223*1.20=1.1655+0.2676=1.4331.

[0237] This calculation process shows that the system automatically uses S with lower free energy (i.e., more reliable performance). phys A higher fusion weight was assigned.

[0238] Step 702, Calculation of the unimodality index M.

[0239] This step exemplifies the calculation process of the unimodality index M in step 402 (online self-calibration) of Embodiment 4.

[0240] Assuming the system is in the short-window microscan phase, local single-peak sharpness enhancement sampling sequences S are acquired around the current focus z=0 at three positions: z=-1, 0, and +1 (unit: focal length step). star (z). To simplify the calculation, assume that the sampled S star The value has undergone logarithmic transformation and light smoothing (corresponding to smooth(log(S))). star (z)))), its sequence is:

[0241] log(S star (-1))=1.80log(S star (0))=2.00(current main peak)log(S star (+1))=1.70.

[0242] The system first calculates the minimum of the second derivative (curvature) of the logarithmic curve of the sequence. A central difference approximation is used here (assuming a step size of s). s For 1):

[0243] d 2 / dz 2 [log(S star (0))]≈(log(S star (+1))-2log(S star (0))+log(S star (-1))) / (s s 2 )d 2 / dz 2 [log(S star (0))]≈(1.70-22.00+1.80) / (1 2 = -0.5. This value of -0.5 quantifies the sharpness of the peak (the more negative, the sharper).

[0244] At the same time, the system detects the secondary peak within the sampling window (or a wider window). Suppose the primary peak is detected at z = +5 with a height of 0.1 times the height of the main peak (corresponding to the ratio of the secondary peak height to the main peak height).

[0245] Suppose the unimodality indicator M is defined as: M = min(second derivative) - η * (secondary peak / main peak ratio) with the weighting factor η (tuned by the system) = 1.0. Then M = -0.5 - 1.0 * (0.1) = -0.6.

[0246] This M value -0.6 will be taken as the target for this round of online self-calibration. The system will next make small updates to the a and g parameters through, for example, mirror gradient steps, and recalculate the M value to make the M value become smaller (e.g., -0.7), i.e., the peak top is sharper (curvature is more negative) or the secondary peak is suppressed lower.

[0247] Embodiment Eight, describes a third layer, i.e., offline or background, self-learning closed-loop mechanism in addition to real-time focusing (as in Embodiments Four, Five).

[0248] Step 801, aggregate the focus process records and real-time sharpness values.

[0249] In this embodiment, the system continuously collects and aggregates the focus process records output by Embodiment Five (e.g., Step 505) at the end of each focus task in the background (e.g., when the CPU / GPU is idle). The focus process record is a structured log. Preferably, the record not only contains the final best focus position, but also includes:

[0250] Performance data: total number of steps and total time taken to reach convergence.

[0251] Trajectory data: complete search trajectory (i.e., S star value corresponding to each z), local curvature estimate of each three-point fitting.

[0252] Event data: count of side lobe events that occurred, number and magnitude of times that side lobe suppression (e.g., g boost) was triggered.

[0253] Quality data: final unimodality indicator M and focus position uncertainty.

[0254] Scene snapshot: scene description vector (from Embodiment Two) at the moment of successful focus.

[0255] Step 802, perform long-term statistical evaluation.

[0256] This step performs statistical analysis on the focus records collected in Step 801 (e.g., the last 1000 times) to uncover potential, systematic performance bottlenecks. For example, the system can perform the following analysis:

[0257] Performance metrics: Calculate the average number of convergence steps, focus success rate (e.g., the proportion of failures not due to side lobes), and timeout failure rate.

[0258] Quality indicators: Calculate the mean and variance of the final unimodality index M, and the mean of the sidelobe ratio.

[0259] Scene correlation analysis: This analyzes the correlation between specific scene description vectors and focus failures (e.g., high sidelobe ratio). For example, through cluster analysis, the system may identify scenes with high anisotropy and low Tf. eff In this scenario, the average sidelobe event count is significantly higher than in other scenarios.

[0260] Step 803: Update model parameters and policy boundaries.

[0261] This step is the core self-learning process in this embodiment. Based on the statistical evaluation results of step 802, the system adaptively updates the hyperparameters or model coefficients used in Embodiments 4 and 5. Specifically, the update may include:

[0262] 1. Update the free energy metric parameter: If step 802 finds that, in a highly anisotropic scenario, S tensor Sidelobe tendency score (from Example 3) i It is systematically underestimated (i.e., S) tensor This leads to a greater number of sidelobe events. The system can update the free energy formula E in Example 4 (step 401) offline. i =a1*(1-M i )+a2Noise i +a3Sidelobe I Among them, S tensor The corresponding coefficient a3 (side lobe penalty) was increased from 1.0 to 1.2.

[0263] 2. Update the boundary of the single-peak enhancement parameter: If step 802 finds that the current average number of convergence steps is too high, and the final peak curvature (the first term of M) is generally too small (the peak is too flat), the system can update the prior value range of the α (peak elevation) parameter in Example 4 (step 402) offline, for example, by setting its lower limit α. min The value was increased from 0.5 to 0.8, which raises the starting point for real-time self-calibration (step 402).

[0264] 3. Update the search strategy boundary: If step 802 finds that false peaks caused by switching the adjacent band of automatic gain control (from Example 6, step 604) account for 30% of all focus failure cases, the system can update the safety width of the avoidance band in Example 5 (steps 502, 503) offline, expanding it from 3 steps to 5 steps.

[0265] Step 804, forming and applying the policy update result.

[0266] The updated parameters (e.g., a3=1.2, a min =0.8, avoidance width=5) are formed into the policy update result. The result is stored and read and applied by the parameter initialization processes of embodiments two, four, five, and six at the next system initialization or calibration cycle.

[0267] In summary, the embodiment enables the method of the application to continuously optimize its internal model parameters through a long-term, background statistical and self-learning closed loop, automatically adapting to device aging (e.g., changes in sensor noise characteristics) or unforeseen specific difficult working conditions, further improving the robustness and long-term performance of the overall focusing system.

[0268] According to an aspect of the present application, constructing the closed-form normalization mapping comprises:

[0269] For the sharpness sub-index, a normalization formula S ~ i =(S i -μ i (σ n 2 )) / σ i (σ n 2 )*2 -b *G -1 is established for scale alignment; wherein μ i and σ i are the mean and standard deviation of the sharpness sub-index S i corresponding to the noise variance σ n 2 , b is the bit depth, and G is the gain energy factor.

[0270] The scale effects of noise, bit depth, and automatic gain on the sharpness sub-index are quantitatively eliminated by the normalization formula to obtain a normalized sharpness sub-index set.

[0271] According to an aspect of the present application, estimating the scene equivalent temperature comprises:

[0272] In the physical parameter estimation and normalization modeling step, the scene equivalent temperature is estimated;

[0273] The scene equivalent temperature is used to calculate the physical gating weight through the Boltzmann distribution, establishing a physical scene adaptive basis for physical gating fusion.

[0274] According to an aspect of the present application, performing data acquisition and basic calibration comprises:

[0275] Adopting scene adaptive modeling based on low-rank plus sparse decomposition, performing response homogenization and black level correction to avoid relying on mechanical shutter;

[0276] Detecting dynamic bad pixels through multi-scale consistency and temporal consistency;

[0277] Adopting structure tensor guided directional interpolation to repair dynamic bad pixels to protect edge structure and generate basic calibration image sequence.

[0278] According to an aspect of the present application, performing data acquisition and basic calibration further comprises:

[0279] Using environmental observation data as radiation anchor points to construct brightness temperature coarse mapping;

[0280] Brightness temperature coarse mapping is used in physical parameter estimation and normalization modeling steps as a physical basis for estimating scene equivalent temperature.

[0281] According to an aspect of the present application, estimating automatic gain function and gain energy factor comprises:

[0282] By locally fitting the gray scale mapping, the segmentation and switching point of the automatic gain are identified;

[0283] Respectively refining fitting for each segment to construct the automatic gain function;

[0284] And calculating the derivative of the automatic gain function and the weighted norm of the gray scale distribution to obtain the gain energy factor.

[0285] According to an aspect of the present application, thermal kernel multi-scale tensor construction and scale entropy clarity calculation can also be:

[0286] According to the scene description vector and the device parameter set. According to the thermal diffusivity κ and the target typical size, a discrete scale set is constructed, and the upper limit is adjusted according to the scene equivalent temperature to avoid excessive smoothing. The thermal kernel scale set is obtained.

[0287] Reading the basic calibration image sequence and the thermal kernel scale set. For each scale s k Generating scale images with thermal kernel convolution, using mirror boundary and energy preserving normalization to obtain scale image stack.

[0288] Based on the scale image stack and the flat area mask. First, do local energy normalization on each scale to weaken the cross-scale intensity bias, then stack according to the spatial axis and the scale axis to form a three-dimensional tensor T(x, y, s), and record the effective voxel mask, to obtain the tensor data structure.

[0289] Read the tensor data structure. Adopt high-order singular value decomposition or Tucker decomposition, select the rank based on the cumulative energy threshold and stability criterion, output each mode factor matrix and core tensor, and obtain the tensor decomposition result.

[0290] Read the tensor decomposition result. Calculate the probability distribution p of the first two singular values λ1, λ2 of the direction energy and the scale mode factor matrix U3 k , to form the hot kernel multi-scale tensor clarity and scale energy spectrum.

[0291] Read the hot kernel multi-scale tensor clarity, normalized transformer set and normalized boundary configuration. Call NAA-Norm-IR to complete scale alignment and obtain normalized hot kernel multi-scale tensor clarity; at the same time, output tensor decomposition auxiliary statistics (direction concentration curve, scale energy spectrum peak position and peak width).

[0292] In summary, the method comprises: acquiring an infrared image sequence, a device parameter set and environment observation data; performing basic calibration to generate a basic calibration image sequence; based on the basic calibration image sequence, the device parameter set and the environment observation data, performing physical parameter estimation and normalized modeling to obtain a normalized clarity sub-index set and a scene description vector.

[0293] The method further comprises: constructing a physically consistent clarity index based on a thermal diffusion-optical defocus combined modulation transfer function, and constructing a hot kernel multi-scale tensor clarity index; based on the free energy metric obtained by evaluating the single-peak prior and noise sensitivity of each index, and combined with the scene equivalent temperature, performing physical gating fusion on the two types of indexes through the Boltzmann distribution to generate a fused clarity; applying a single-peak enhancement transformation to the fused clarity, and performing a short window micro-scan to calculate a single-peak index, and maximizing the index to online self-calibrate the transformation parameters, and finally outputting a single-peak enhanced clarity curve.

[0294] The present application can also perform focus search through three-point fitting and curvature adaptive step, and when the sidelobe risk is detected, dynamically write back the sidelobe suppression parameter to reshape the curve shape, solving the problems of traditional index scale inconsistency, easy to be disturbed by gain and poor curve shape.

[0295] First, for the scale variability and false peak problem caused by automatic gain control (AGC) switching, the present scheme quantitatively eliminates the scale influence of gain, bit depth and noise variance on the clarity index through the construction of a closed normalized mapping. At the same time, through the linkage of automatic gain segmentation identification and subsequent search strategy, the search algorithm can actively avoid the gain switching band and avoid the interference of false peaks.

[0296] For the shape non-idealness (i.e. sidelobe and wide flat top) caused by traditional empirical operators, the present scheme solves it in multiple layers:

[0297] The physical consistent definition and the thermal kernel multi-scale tensor definition based on energy concentration degree replace the traditional empirical operator, and the single-peak trend is ensured to be better.

[0298] Through physical gating fusion, the free energy of each index (including side lobe tendency and noise sensitivity evaluation) is adaptively weighted to dynamically select and combine the most ideal index in the current scene.

[0299] Single-peak enhancement transformation and online self-calibration are adopted to actively sharpen and suppress side lobes of the fused curve to maximize a calculable single-peak index, and the curve shape is actively reshaped.

[0300] Through side lobe detection and reverse linkage mechanism, a cooperative closed loop of measurement and search is established. When the search algorithm detects the side lobe risk, the suppression parameter can be dynamically written back to the measurement module, so that the definition curve actively deforms to facilitate the search, and the problem of the separation of measurement and control in the traditional scheme is solved.

[0301] The preferred embodiments of the present application are described in detail above, but the present application is not limited to the specific details in the above embodiments, and various equivalent transformations of the technical solutions of the present application can be made within the technical concept of the present application, and these equivalent transformations all belong to the protection scope of the present application.

Claims

1. A multi-dimensional complementary sharpness evaluation method for infrared images, characterized in that, include: Acquire infrared image sequences, equipment parameter sets, and environmental observation data; Perform basic calibration on the infrared image sequence to generate a basic calibration image sequence; Based on the basic calibration image sequence, equipment parameter set and environmental observation data, physical parameter estimation and normalization modeling are performed to obtain a set of normalized sharpness sub-indices and scene description vectors; Based on the normalized set of sharpness sub-indices and the scene description vector, multi-dimensional complementary sharpness construction and unimodal enhancement are performed to produce a unimodal enhanced sharpness curve and real-time sharpness value; This includes performing multi-dimensional complementary clarity construction and unimodal enhancement: Construct a physical consistency clarity metric; Construct a clarity index for multiscale tensors of thermonuclear kernels; Physical gating fusion is applied to the two types of indicators to generate fused clarity. A single-peak enhancement transformation is applied to the fused sharpness and online self-calibration is performed, producing a single-peak enhancement sharpness curve and real-time sharpness values.

2. The method according to claim 1, characterized in that, Performing physical gating fusion includes: Virtual defocus perturbation and noise injection are applied to the basic calibration image sequence to evaluate the unimodality prior and noise sensitivity of the two types of indicators, and to construct their respective free energy measures. Extract the scene equivalent temperature from the scene description vector; The physical gating weights are calculated using the Boltzmann distribution based on the free energy metric and the scene's equivalent temperature. Physical gating weights are used to weight the two types of indicators to generate fusion clarity.

3. The method according to claim 1, characterized in that, Perform online self-calibration, including: Perform a short-window micro-scan around the current focus to acquire a local single-peak enhanced sharpness sampling sequence; The unimodality index of a local sampling sequence is calculated by determining the minimum value of the second derivative of the logarithmic curve of the sequence and combining it with the ratio of the secondary peak height to the primary peak height. With the goal of maximizing the unimodality index, the parameters of the unimodal enhancement transform or the weights of the physical gated fusion are updated online. Based on the updated parameters or weights, the final single-peak enhanced sharpness curve and real-time sharpness value are generated.

4. The method according to claim 1, characterized in that, Construct a physically consistent clarity metric, including: A frequency domain weighting function is constructed based on the synthesized modulation transfer function, which includes optical defocus variance and thermal diffusion equivalent time. The amplitude spectrum is calculated for the basic calibration image sequence, and a frequency domain weighted integral is performed using a weighting function to obtain the original physical consistent sharpness; Obtain a set of normalization transformers from the physical parameter estimation and normalization modeling steps, and call them to scale the original physical consistency sharpness.

5. The method according to claim 1, characterized in that, Construct a multi-scale tensor clarity index for heat kernels, including: The set of heat core scales is determined based on the scene description vector; A three-dimensional tensor is constructed by convolving the basic calibration image sequence with a heat kernel scale set. Perform tensor decomposition on the three-dimensional tensor to extract directional energy singularities and scale modulus factor matrices; Based on the directional energy singularity and scale modulus factor matrix, the directional energy concentration and scale energy concentration are calculated to form the clarity of the original thermonuclear multiscale tensor. Obtain a set of normalization transformers from the physical parameter estimation and normalization modeling steps, and call them to scale the clarity of the original thermonuclear multiscale tensor.

6. The method according to claim 1, characterized in that, The method also includes: By utilizing a single-peak sharpness enhancement curve and real-time sharpness values, a focus search and control closed loop is executed to determine the optimal focus position; The focus search and control closed loop includes performing three-point fitting, performing sidelobe detection, and generating a focus drive sequence; The sidelobe detection process includes: During the focus search process, the presence of sidelobe risk is determined based on the inconsistency between the ratio of secondary peaks to primary peaks or curvature. In response to sidelobe risk, an increment of the sidelobe suppression parameter is generated; The sidelobe suppression parameter increments are written back to the multi-dimensional complementary sharpness construction and unimodal enhancement steps to dynamically change the shape of the unimodal enhanced sharpness curve in subsequent sharpness calculations.

7. The method according to claim 6, characterized in that, Perform a three-point fit, including: Collect real-time sharpness values ​​corresponding to the focus position within the search range; A parabola is fitted between the focus position and the real-time sharpness value to obtain the candidate focus position and local curvature estimate; Based on local curvature estimation, a step size suggestion for the next movement is set to achieve curvature adaptive step size.

8. The method according to claim 7, characterized in that, Generate the focus drive sequence, including: Obtain the lens mechanism model from the device parameter set; Based on candidate focus positions or step size suggestions, trajectory planning with jerk constraints is used to generate an initial driving sequence; By combining the lens mechanism model, reverse clearance compensation and feedforward friction compensation are performed on the initial drive sequence to generate the final focus drive sequence.

9. The method according to claim 2, characterized in that, Performing physical parameter estimation and normalization modeling includes: Estimate the noise variance, automatic gain function and gain energy factor, and bit depth; Estimate the scene equivalent temperature corresponding to the environmental observation data; Construct a closed-loop normalized mapping to generate a set of normalized clarity sub-indices, and aggregate scene equivalent temperatures to generate scene description vectors.

10. The method according to claim 1, characterized in that, Perform data acquisition and basic calibration, including: Adaptive scene modeling based on low-rank sparse decomposition is employed to perform response uniformization and black level correction in order to avoid reliance on mechanical shutter. Dynamic bad pixels are detected through multi-scale consistency and time consistency. Directional interpolation guided by structural tensor is used to repair dynamic bad pixels, thereby protecting edge structures and generating a basic calibration image sequence.

Images