A dynamic weight optimized high-noise infrared thermal image sequence registration method

Abstract

The application discloses a dynamic weight optimized high-noise infrared thermal image sequence registration method, relates to the field of infrared thermal imaging nondestructive testing and image registration, and particularly relates to a three-dimensional registration method for high-noise infrared thermal image time sequence data of a complex surface sample. The method combines artificial feature point calibration and algorithm dynamic weight optimization, realizes accurate alignment of infrared thermal image sequences acquired at long time intervals, and belongs to the field of image processing and infrared monitoring technology. The application aims to overcome the above defects of the prior art, provide a man-machine interactive image stabilization registration method based on cross-time span high-noise infrared images, and solve the problem that long-interval infrared images are difficult to automatically and accurately align. Through man-machine interaction, selection of key points, a multi-source image stabilization processing framework and a dynamic weight optimization algorithm, the application realizes robust registration and alignment between infrared images acquired by different devices in different years.

Description

Technical Field

[0001] This invention relates to the field of infrared thermal imaging nondestructive testing and image registration, and particularly to a three-dimensional registration method for high-noise infrared thermal image time series data of complex surface samples. This method combines manual feature point calibration with dynamic weight optimization of the algorithm to achieve accurate alignment of infrared thermal image sequences acquired over long time intervals, belonging to the fields of image processing and infrared monitoring technology. Background Technology

[0002] Infrared images have wide applications in industrial non-destructive testing, remote sensing monitoring, and equipment inspection. However, infrared images acquired at different times often exhibit significant differences due to variations in the year of capture, environmental conditions, and imaging equipment performance. For example, infrared thermal images from different periods may differ in resolution, noise levels, contrast, and scene thermal characteristics. Furthermore, infrared imaging is frequently affected by noise, especially in images acquired with older equipment or in harsh environments. These factors make accurate registration (alignment) of infrared images spanning different years and environments extremely difficult.

[0003] Traditional image registration methods (such as algorithms based on automatic feature point extraction) perform poorly on noisy infrared images. On one hand, high noise can obscure or distort detailed features in infrared images, making it difficult for automatic feature point detection algorithms to reliably extract the corresponding key points. On the other hand, due to changes in the environmental background, some previously obvious thermal features may change or even disappear in infrared images from different periods, leading to erroneous correspondences in automatic matching. Simply relying on automatic algorithms is susceptible to feature drift and low signal-to-noise ratios, necessitating the introduction of human intervention and more robust optimization strategies. While manual alignment can utilize the human eye to identify key features, the purely manual approach is inefficient and its accuracy is affected by human factors.

[0004] Therefore, a new registration method is urgently needed that can combine manual point selection with automatic optimization algorithms to achieve robust and high-precision stable image registration between infrared images with high noise and spanning multiple time periods. By introducing human-computer interaction to obtain reliable initial control point pairs and using algorithms to dynamically optimize and adjust them, the shortcomings of existing technologies can be overcome, and the stability and accuracy of registration can be improved. Summary of the Invention

[0005] The purpose of this invention is to overcome the aforementioned shortcomings of the prior art and provide a human-computer interactive image stabilization and registration method based on high-noise infrared images spanning a long time span, thereby solving the problem of difficulty in automatically and accurately aligning long-interval infrared images. By combining human-computer interaction for key point selection, a multi-source image stabilization processing framework, and a dynamic weight optimization algorithm, robust registration and alignment between infrared images acquired from different years and different devices is achieved. To achieve the above-mentioned objective, this invention provides the following technical solution: A human-computer interactive image stabilization and registration method, applicable to high-noise infrared images spanning a long time span, comprising the following steps:

[0006] Step 1: Obtain at least two infrared sequences of the same target object from different time periods (this invention is an image stabilization technique, which has advantages in situations where it is impossible to accurately find the accurate three-dimensional angle. The longer time span can significantly amplify environmental / equipment differences and thermal feature drift, so that the robustness of the proposed dynamic weight and human-computer interaction framework can be fully verified and utilized) as the registration dataset and perform simple preprocessing.

[0007] The acquired infrared sequences may come from different devices or different environmental conditions, and their noise levels are high. The images contain the same target or scene area to be aligned.

[0008] Step 2: Using at least two infrared images obtained in Step 1, one infrared image is used as the reference image, and the other infrared images are used as the images to be registered. Based on prior knowledge of the target object's 3D structure and thermal features, the user interactively selects several corresponding control point pairs on the two images. Each pair of control points corresponds to the same physical feature location in both the reference image and the image to be registered. Control point pairs can be selected from structural points with obvious features in the images, such as corner points, edge intersections, or the center point of thermal anomalies.

[0009] Step 3: When the number of selected control point pairs is not less than four, the perspective transformation matrix H is calculated using a rigid registration algorithm based on the projection model. The image at one time moment is projected and transformed into the coordinate system of the image at another time moment to achieve initial alignment. The perspective transformation matrix H has eight degrees of freedom and can represent the comprehensive distortions such as translation, rotation, scaling and perspective projection caused by changes in the camera's viewing angle.

[0010] Using the initially selected set of control points, the perspective transformation matrix is ​​solved by the weighted least squares method, and the currently registered image is overlaid and displayed in real time on the human-computer interaction interface.

[0011] Users can intuitively view the degree of overlap and error distribution between two images, and the system simultaneously provides quantitative feedback indicators of registration error (such as overlap deviation value and color error map). Through this real-time feedback mechanism of human-computer interaction, researchers can immediately observe the results and correct errors in a timely manner during the registration process, thereby significantly improving the accuracy of infrared image registration for irregular surfaces. This combination of manual intervention and real-time feedback is particularly suitable for scenarios where automatic algorithms struggle to achieve high-precision results due to factors such as high image noise and changes in the field of view.

[0012] Step 4: Dynamic Weight Optimization Stage. A key feature of this invention is the introduction of a layer-by-layer weighting strategy during the registration optimization process. This involves dynamically iteratively adjusting the weights of the control point pairs to improve image stabilization robustness.

[0013] Step 4, the "dynamic weight optimization stage", is seamlessly connected with Step 2, the "interactive point selection", and Step 3, the "initial H solution".

[0014] After generating control points in step 2 and obtaining the initial H in step 3, step 4 is used to dynamically optimize the weights. Although the operation process is the same, the initial weights in step 4 come from steps 2 and 3.

[0015] The weights in the point selection stage are calculated based on steps 2 and 3, which are the initial stages. Step 4 is the iteration stage. The system assigns an initial weight to each pair of control points and supports manual adjustment. The weight depends on the confidence level of the point pair and its contribution to the registration. For example, control point pairs located in regions with drastic temperature gradient changes (such as crack tips, material corners, decorative vertices, etc.) contain rich feature information and are given higher initial weights; while point pairs located in regions with uniform temperature distribution or susceptible to artifact interference are given lower weights.

[0016] After obtaining the initial perspective transformation, the system calculates the reprojection error of each control point, i.e., the positional deviation between the transformed point and its corresponding actual point. Residual evaluation and weight update: control point pairs with smaller errors maintain higher weights, while points with larger errors have their weights reduced exponentially. A residual-driven exponential decay model is used to update the weights, ensuring that outlier point pairs with high errors do not excessively interfere with the calculation of the transformation matrix, while not completely eliminating the information of these point pairs.

[0017] Step 5: In multiple iterations, the system continuously executes a cycle of "calculation transformation - residual evaluation - weight update," that is, refining the weight distribution layer by layer. To balance the influence of manual prior knowledge and automatic optimization, a convergence decay factor is set to gradually reduce the proportion of initial weights, allowing data-driven weights to gradually become dominant as iterations progress. Specifically, early iterations retain a certain proportion of the influence of manual initial weights to prevent the algorithm from prematurely weakening the weights of key points; while in later iterations, the weights are more adaptively adjusted based on the residuals, so that the final solution converges to the true error distribution. This layer-by-layer weight fusion strategy ensures that the algorithm fully utilizes the anchor points specified by expert experience while automatically correcting deviations based on data, taking into account both prior knowledge and algorithm robustness.

[0018] Step 6: After the weight iterative optimization reaches the convergence condition, the system determines the final perspective transformation matrix and the weights of each control point, applies the registration transformation to the image in real time, and outputs the transformation matrix, allowing users to perform global registration of the entire infrared sequence (registration is to register two 2D photos with obvious features after dimensionality reduction from two 3D datasets. The output transformation matrix allows users to register each frame of the 3D dataset. The implementation process is self-evident to those skilled in the field).

[0019] As a preferred embodiment, in step 1 of the present invention, the infrared sequence image obtained at an earlier time is used as the reference image, and the infrared sequence image obtained at a later time is used as the image to be registered.

[0020] As another preferred option, in step 1, the preprocessing uses Fast Fourier Transform (FFT) bandpass denoising to reduce noise in the thermal image sequence and reduce the impact of environmental noise; and the principal component analysis method is used to perform dimensionality reduction on the sequence, extracting the main features of the three-dimensional time series data and mapping them onto the two-dimensional image.

[0021] It is worth noting that when the difference in the infrared camera model leads to a large difference in the image, especially when the use of an additional wide-angle lens causes obvious nonlinear optical distortion in the image, optical distortion correction should be performed on the image according to the intrinsic parameters of the infrared camera used, in order to eliminate the nonlinear distortion caused by lens distortion.

[0022] As another preferred option, the main feature is: performing PCA on the time series, taking the top K principal components with a cumulative variance contribution rate ≥ 80%, and using K = 1 as the feature image by default; if < 80%, taking K = 2 and performing weighted synthesis.

[0023] As another preferred option, in step 2, the user interactively selects several corresponding control point pairs on the two images based on prior knowledge of the three-dimensional structure and heat map features of the target object. The specific physical feature positions corresponding to each pair of control points in the reference image and the image to be registered are as follows: gradient magnitude at the corresponding point ≥ G0 (default 10–30 gray levels / pixel), corner response ≥ C0 (default Harris ≥ 0.01), and local texture variance ≥ V0 (default ≥ 15). The values ​​in parentheses are quantification criteria used to concretize the aforementioned "control point selection principle" and ensure reproducibility. The default threshold can be adjusted according to the working conditions.

[0024] As another preferred option, in step 3, the perspective transformation matrix H adopts the conventional planar homography perspective transformation matrix H (3×3), which has 8 degrees of freedom while maintaining the same scale, and can simultaneously represent translation / rotation / scaling and perspective. This document refers to it as the "perspective transformation matrix H".

[0025] As another preferred option, the initial weights automatically assigned by the system during the iteration in step 4 can be assigned not only automatically by the reprojection error, but also by taking into account the degree of temperature change at that point within the entire infrared sequence. However, the adaptability of this preferred option under different operating conditions should be considered. That is, the three weights can be modified according to the infrared sequence. For example, under the condition of very uniform temperature change, the influence of β3 on the initial weights can be eliminated.

[0026] Initial weights:

[0027] Where C i For Harris corner / edge response, V i For the 11×11 neighborhood texture variance, T i This represents the temperature standard deviation of the pixel over time; the default values ​​are β1 = 0.5, β2 = 0.3, β3 = 0.2, and w... i (0) Normalize to [0.1, 1.0].

[0028] Dynamically updated:

[0029]

[0030] The default α = 1.0, and the residual MAD × 1.4826 is used. min =0.05,w max =1.2.

[0031] As another preferred option, a cubic interpolation algorithm is used to quickly redraw the registration result frame during the interaction and iteration process, ensuring smooth operation and no obvious image distortion.

[0032] As another preferred embodiment, in step 5 of this invention, the initial artificial weight is:

[0033]

[0034] η k =min(1,k / K0)

[0035] The default value is K0=8. That is, in the first 8 iterations, the system gradually transitions from prior knowledge to data-driven decision-making.

[0036] As another preferred embodiment, the present invention introduces a key point weight freezing protection mechanism in step 5: for user-annotated high-confidence key control points, if the error increases during iteration, the system limits the reduction in the weight of that point, for example:

[0037] Set a lower limit for high-confidence control points: When its residual satisfies T = 3 consecutive times Unfreeze and return to regular updates;

[0038] This avoids excessive weakening of the weights of important anchor points due to temporary noise. Through the above dynamic weight allocation and optimization strategy, the framework of this invention achieves a balance between manual intervention and automatic optimization, significantly enhancing the robustness of the registration process in the context of feature drift and high noise.

[0039] Secondly, in the process of combining traditional manual registration and automated registration in step 5, the manually selected control points may be affected by some mismatches. Therefore, this invention proposes an automatic error correction mechanism (this mechanism is simplified in the following detailed implementation; it means that after calculating the reprojection error in step 133, for larger reprojection errors, the outlier points (e...) are corrected). i >3σ) Enables local short-range position optimization. This mechanism automatically identifies and corrects erroneous control point pairs during registration through reprojection error detection and control point optimization algorithms. The system selects obviously abnormal control point projection errors based on reprojection errors, proposes an optimal short-range position offset adjustment scheme, and allows for manual supervision to determine whether to perform the offset. This intelligent correction mechanism effectively reduces systematic errors caused by high-weight points in manual intervention pairs and further improves registration accuracy.

[0040] For the convergence condition in step 6, the convergence condition can be met if any one of the following three conditions is met: the weighted residual change rate of all control points is lower than the threshold, the update magnitude of the transformation matrix is ​​lower than the threshold, or manual stopping is required. This can minimize the efficiency of manual registration and avoid unnecessary waste of human resources.

[0041] It is worth noting that this invention primarily targets field-of-view registration of thermal imaging sequences for complex surface samples, and is particularly suitable for registering datasets after long-term intervals of observation. When the sample itself undergoes significant deformation, this invention cannot provide accurate registration results.

[0042] Furthermore, the correction mechanism described in this invention is as follows: automatic error correction is embedded in the residual evaluation and weight update in step 5: for outliers (e i >3σ) Enable local short-distance position optimization (maximum displacement r) max =3 pixels, subpixel search), if corrected e i If the value is ≤2σ, retain it; otherwise, reduce the weight to w. min .

[0043] Beneficial effects of this invention:

[0044] This invention overcomes the perspective changes caused by long intervals and differences in imaging from different devices by introducing a perspective projection model and distortion correction, achieving accurate three-dimensional alignment of infrared data over a long time span. Even with significant variations in observation conditions, the registered dataset maintains excellent detail consistency, and thermal anomaly locations are accurately matched. This method effectively addresses the problem of data instability during long-term monitoring, significantly improving the accuracy and reliability of infrared nondestructive testing on complex samples.

[0045] This invention introduces a dynamic weighting strategy, automatically adjusting the weights of control point pairs based on the residual magnitude. Compared to traditional methods that simply discard points with large errors, this invention reduces their weights but does not completely discard them, allowing these points to still contribute to the final result. Simultaneously, the weights of control point pairs with low residuals are increased, ensuring that reliable point pairs dominate the optimization process. This strategy improves the algorithm's robustness to outliers; even if the initial control point pairs have some errors, the optimization process can still converge to accurate alignment, avoiding registration failures due to individual mismatches.

[0046] This invention's method is highly adaptable to image noise, differences in imaging equipment, and variations over time. Whether using infrared images from multiple years or data from different infrared cameras, this method can register images as long as common scene features exist. Dynamic weight optimization ensures stable alignment even for noisy and low-contrast infrared images; the human-computer interaction mechanism guarantees that registration can still be completed with human recognition when automatic algorithms are ineffective. This gives this method significant advantages in long-term monitoring of infrared image sequences and historical data comparison, filling the gap in existing technologies for infrared image stabilization under complex conditions. Attached Figure Description

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The scope of protection of the present invention is not limited to the following description.

[0048] Figure 1 This is a schematic diagram of the overall process of a dynamic weight optimization method for high-noise infrared thermal image sequence registration according to the present invention.

[0049] Figure 2 This is a schematic diagram illustrating the manual selection of corresponding feature points in the method of the present invention.

[0050] Figure 3 This is a schematic diagram showing an overview of the results of registering an infrared image of an ancient bronze helmet using the method of the present invention.

[0051] Figure 4 This is a schematic diagram showing the comparison of the selected region of interest (ROI) before and after registration using the method of the present invention. Detailed Implementation

[0052] This invention provides a human-computer interactive image stabilization and registration method, applicable to the precise alignment of infrared image sequences containing high noise across time spans, different imaging devices, and especially for target objects with complex surfaces. (See reference...) Figure 1 The method of the present invention includes the following main functional modules: data preprocessing module (11), human-computer interaction and dynamic weight optimization module (including 12 and 13) and registration result generation module (14).

[0053] Module 11: Data Preprocessing Module. This module is responsible for performing preliminary processing on the input raw infrared thermal image sequence, laying the foundation for subsequent accurate registration.

[0054] First, acquire at least two sets of infrared thermal image sequences of the same target object acquired at different times or under different observation conditions. One set serves as a reference image, whose homogeneous coordinates of feature points can be represented as P1; the other set serves as the image to be registered, whose corresponding homogeneous coordinates of feature points can be represented as P2.

[0055] 111 Principal Component Analysis Dimensionality Reduction: Principal component analysis is used to reduce the dimensionality of infrared thermal imaging time series data and extract two-dimensional feature images that can represent the main thermal features of the original sequence.

[0056] 112 Fast Fourier Transform Noise Reduction (i.e., the "Preprocessing" part of "Step 1" in the "Summary of the Invention"): Filtering techniques such as... Figure 1 The example uses Fast Fourier Transform (FFT) noise reduction. Here, the specific noise reduction method can be adjusted according to the specific working conditions (different preprocessing noise reduction methods are used for noise under different working conditions, which is a common technique in this field) to denoise the image sequence and improve the signal-to-noise ratio.

[0057] 113 Radial optical distortion elimination (optional): When the input data contains significant optical distortion caused by the camera lens, distortion correction is performed based on camera intrinsic parameters (distortion correction usually uses the Brown-Conrady model, which is a conventional technique in this field).

[0058] Modules (including 12 and 13): Human-computer interaction and dynamic weight optimization module. This module is the core of the invention. It closely integrates intelligent human point selection with dynamic optimization of the algorithm in an interactive iterative loop.

[0059] 12 Human-Computer Interaction Sub-modules:

[0060] This module is the core of the invention. It closely integrates manual intelligent point selection with dynamic optimization of the algorithm in an interactive iterative loop. The goal is to solve for the optimal perspective transformation matrix H, so that point P1 on the reference image can be precisely aligned with the corresponding point P2 on the image to be registered after transformation, that is, satisfying the relationship P2≈H*P1.

[0061] 121 Manually mark ≥4 pairs of control points: Refer to Figure 2 The user's preprocessed reference image ( Figure 2 On the left, the area marked 2101 (where 2102 is the object outline, and 2103 to 2108 are selected example control point pairs) and the image to be registered ( Figure 2 On the right, in the area marked 2201 (where 2202 is the object outline and 2203 to 2208 are the corresponding example control point pairs), at least four pairs of corresponding control points are interactively selected. These points should correspond to the same physical feature positions on the target object.

[0062] 122 Real-time Transparent Overlay Preview of Registration Effect: Once the number of selected control point pairs meets the condition for calculating the initial transformation (usually ≥4 pairs), the system immediately enters the internal 13 Dynamic Weight Optimization Iteration Submodule. The perspective transformation matrix calculated by this submodule (whether initial or in iterative optimization) is used in real time to transform the image to be registered and overlay it semi-transparently onto the reference image for user observation. Users can use this real-time preview effect to return to 121 to adjust, delete, or add control point pairs at any time; this adjustment will immediately trigger a new round of 13 Dynamic Weight Optimization Iteration.

[0063] 13 Dynamic Weight Optimization Iteration Submodule (as the core of the internal loop of the human-computer interaction module): This submodule is activated after the user completes a control point operation (adding, deleting, adjusting points, or automatically triggered by the system) and performs iterative optimization until internal convergence is achieved or the user performs a new interaction operation.

[0064] 131 Initial Weight Assignment: Assign initial weights to the current set of control point pairs provided by 121. This can be user-specified, system-assisted in setting based on certain characteristics, or defaulted to equal weights.

[0065] 132 Calculate the perspective matrix: based on the current control point pairs and their weights w k i (where k is the current iteration number), the perspective transformation matrix H is solved by minimizing the weighted projection error E(H) in real time using, for example, the Levenberg-Marquardt algorithm. k The weighted projection error can be expressed as: E(H)=Σ i w k i *||H*P 1i -P 2i || 2 +λ||H|| F 2 Where, Σ i This represents the summation over all i control point pairs, ||H*P 1i -P 2i || 2 H represents the square of the projection error between the i-th pair of control points, λ is the regularization parameter, and ||H|| F 2 H is the Frobenius norm of the perspective transformation matrix H, used to constrain model complexity to prevent overfitting.

[0066] 133 Calculate reprojection error: Using the currently calculated H, calculate the reprojection error for each control point pair.

[0067] 134-weighted exponential decay update: based on reprojection error e k i The weights are automatically updated using a residual-driven exponential decay model. The new weights w k+1 i It can be calculated by the following formula: w k+1 i =w k i *exp(-α*(e k i ) 2 Where α is a positive convergence decay factor used to control the exponential sensitivity of the weights to the error. Error e k i The larger the point, the higher its weight w. k+1 oThe more it decays, the better. After calculation, the automatically adjusted results are fed back to 122 in real time for preview. This design emphasizes that dynamic weight optimization is a real-time response and internal optimization process in the human-computer interaction process. Every time the user makes an adjustment, the system will quickly perform one or more rounds of weight optimization iterations and feed back the latest optimization results to the user.

[0068] 135 Weight Freeze: Protects high-confidence key points marked by users during weight updates to prevent their weight from being excessively weakened. Figure 1 The dashed line indicates that weight freezing may affect the final result of the weight exponential decay update or serve as a judgment condition. A dashed threshold protection is set for high-confidence keypoints annotated by users. If the weight is lower than the threshold during the exponential decay update, it is truncated to that value, and touching the threshold is used as an anomaly judgment condition to avoid excessive weakening of keypoint weights due to temporary noise.

[0069] 136. Convergence Criterion: Determines whether the iteration satisfies the convergence condition. The convergence condition may include at least one of the following:

[0070] a) Stabilization of weighted residuals: the sum of squared weighted residuals Σw for all control point pairs i (e i ) 2 The change is less than the preset threshold ε res .

[0071] b) Stabilization of the transformation matrix H: The transformation matrix H calculated in the current iteration k The transformation matrix H of the previous iteration k-1 The difference between them (e.g., the Frobenius norm) is less than a preset threshold ε H .

[0072] c) Reaching the preset maximum number of iterations N max .

[0073] d) Manual termination.

[0074] If convergence is not achieved: return to 121 and continue the current optimization iteration.

[0075] If convergence is achieved: the result of the perspective transformation matrix H obtained from the current optimization is fed back to 14 and the registration result is generated.

[0076] Module 14: Generate Registration Results. The system will output the perspective transformation matrix finally optimized by Module 13. This matrix is ​​used to perform geometric transformation on the original infrared image to be aligned (or its preprocessed feature image) to generate a final registered image that is precisely aligned with the reference image.

[0077] This invention achieves real-time, adaptive registration optimization during user interaction by tightly integrating dynamic weight optimization iterations into the human-computer interaction module, thereby improving operational efficiency and final registration accuracy.

[0078] Example:

[0079] This embodiment aims to illustrate the specific application of the dynamic weight optimization method for registering high-noise infrared thermal image sequences according to the present invention. This method selects key points through human-computer interaction and combines them with a dynamic weight optimization algorithm to achieve robust registration of high-noise infrared images with a long time span. The experimental object is a century-old ancient bronze helmet located in Italy. The sample surface is relatively complex. Infrared thermal image sequence data of the helmet were acquired in 2007 (as the reference image source) and 2024 (as the image source to be registered).

[0080] Step 1: Obtain and preprocess the dataset to be registered (corresponding to) Figure 1 The data preprocessing module (11) acquires infrared thermal image sequences of the ancient bronze helmet from 2007 and 2024. The 2007 sequence is used as the reference dataset, and the 2024 sequence is used as the dataset to be aligned. In the data preprocessing module (11), principal component analysis (PCA) is first performed on both sequences to reduce dimensionality (111), extracting two-dimensional images containing the main thermal features. Subsequently, fast iterative filtering (FIF) is used to denoise the two two-dimensional feature images (112). Considering that different cameras may have been used for the two images, radial optical distortion correction (113) is performed on the images according to the intrinsic parameters of each camera to ensure geometric accuracy. After processing, the reference feature image from 2007 and the feature image to be aligned from 2024 are obtained.

[0081] Step 2: User interactively selects control point pairs (starting from) Figure 1 Human-Computer Interaction Submodule 121) In the human-computer interaction submodule (12), the operator refers to Figure 2 As shown, in the preprocessed reference feature image ( Figure 2 Left 2101 region) and feature image to be aligned ( Figure 2 On the right side (region 2201), based on prior knowledge of the helmet's three-dimensional structure and thermal features, four pairs of initial corresponding control points (121) were selected. These control points were selected at easily identifiable and relatively stable physical feature locations such as the helmet's top ornament and brow bone edge.

[0082] Step 3: Calculate the initial perspective transformation and provide real-time feedback (some functions are provided by...). Figure 1(The human-computer interaction submodule 122 and the dynamic weight optimization iteration submodule 132 work together to complete this.) When the number of selected control point pairs reaches 4 pairs (not less than four pairs), the system immediately performs initial weight allocation (131) for these 4 pairs of control points within the dynamic weight optimization iteration submodule (13) (for example, the initial weight of all points is set to 1.0). Subsequently, based on these initial points and their weights, the initial perspective transformation matrix H is calculated using the weighted least squares method (132). The result of the calculated initial perspective transformation matrix H is used by the human-computer interaction submodule (12) to transform the feature image to be aligned and displays it on the reference feature image in real time in a semi-transparent manner (122) for the operator to intuitively view the initial alignment effect.

[0083] Step 4: Dynamic weight adjustment (the core is in) Figure 1 After observing the initial preview effect, the operator in the dynamic weight optimization iteration submodules (133, 134, 135) found that the alignment of some areas was not ideal. At this time, the operator returned to the human-computer interaction submodule (121) to select another 11 pairs of control points, bringing the total number of control point pairs to 15. For these 15 pairs of control points, the system re-allocated the initial weights in the dynamic weight optimization iteration submodule (13) (131), of which 5 pairs were marked by the operator as high-confidence key points and assigned higher initial weights. The system began iterative optimization:

[0084] First, based on the current 15 pairs of points and their weights, the perspective transformation matrix H(132) is recalculated.

[0085] Then, the reprojection error e(133) for each control point pair is calculated.

[0086] Next, based on the reprojection error, an exponential decay model w is adopted. new =exp(-α*e 2 )*w old (α is set to 0.1) The weights of each control point are automatically adjusted (134). The weights of points with larger errors are significantly reduced.

[0087] For the five pairs of control points marked as key points, a weight freeze protection mechanism (135) is enabled to limit their weights from being excessively weakened due to temporary error fluctuations.

[0088] Step 5: Iterative optimization and weight refinement (corresponding to) Figure 1The dynamic weight optimization iteration submodule 13 (loop and 136 judgment) continuously executes the loop of "calculate transformation (132) - evaluate residual (133) - update weight (134, 135)". After each iteration of weight update, the new transformation result is fed back to the operator for preview in real time through the human-computer interaction submodule (122). During the preview, the operator may return to the human-computer interaction submodule (121) to fine-tune some control point pairs or delete obviously erroneous control point pairs. Each such user interaction triggers the dynamic weight optimization iteration submodule (13) to re-optimize based on the current set of control point pairs and weights. In order to balance the influence of manual prior and automatic optimization, a convergence decay factor is introduced in the iteration process to gradually make data-driven (residual-based) weight adjustment dominate, while ensuring that the key point weights manually specified in the early iterations are retained.

[0089] Step 6: Achieve convergence and output the final result (from...) Figure 1 The dynamic weight optimization iteration submodule 136 judges and sends the result to the registration result generation module 14). After several rounds of manual interactive adjustment and dynamic weight optimization iteration within the system (for example, a total of 5 major manual adjustments were performed, and an average of 15 weight optimization iterations were performed within the system after each adjustment), the convergence judgment (136) of the dynamic weight optimization iteration submodule (13) indicates that the iteration has reached a stable state. Specifically, the weighted residual change rate of all control points is lower than the preset threshold of 0.005, and the parameter update amplitude of the perspective transformation matrix is ​​also lower than the threshold of 10. -6 The operator also confirms the current registration effect through real-time preview (122). At this point, the system determines the final perspective transformation matrix H. final And the final weights of each control point. This H... final The images are sent to the registration result generation module (14). This module uses the final transformation matrix to perform a geometric transformation on each frame of the original infrared thermal image sequence from 2024, resulting in an image sequence that is spatially precisely aligned with the reference infrared thermal image sequence from 2007.

[0090] Results verification and presentation: Refer to Figure 3 This figure shows an overview of the registration results obtained after applying the method of this invention. The figure contains several sub-figures:

[0091] Image 3001 shows the original reference image (a helmet image from 2007).

[0092] Image 3002 shows the original image to be registered (a helmet image from 2024), which shows differences in viewpoint and detail compared to the reference image.

[0093] Image 3003 also displays the original image to be registered (the helmet image from 2024), but the purpose of this sub-image is to provide a visual alignment reference with reference image 3001 and the final registered image 3004. Typically, it is aligned with 3001 and 3004 in the layout horizontally and vertically using grid lines to facilitate observation of the changes before and after the transformation.

[0094] Image 3004 shows the effect of overlaying (or displaying side-by-side) the image to be registered with the reference image after transforming it using the final perspective transformation matrix calculated by the method of this invention. It can be seen that the transformed image and the reference image achieve good alignment in terms of contours and main features.

[0095] Further reference Figure 4 This figure shows a detailed comparison before and after registration on several selected regions of interest (ROIs). Each row represents one ROI:

[0096] The left column shows the selected ROI regions in the original image to be registered.

[0097] The middle column displays the corresponding ROI regions in the reference image.

[0098] The right column shows the alignment of the ROI region in the image to be registered with the corresponding ROI in the reference image after registration transformation using the method of this invention. By comparing the middle and right columns, it can be clearly observed that even in local details, the method of this invention can achieve accurate feature alignment, such as subtle textures and the edges of erosion spots; the arrows indicate these corresponding features.

[0099] The registered image sequence was visually inspected, and as... Figure 3 and Figure 4 The registration results and ROI detail comparison shown demonstrate that the method of this invention can effectively compensate for image differences caused by long-term spans and changes in viewpoint, achieving high-precision alignment of the helmet's main outline, key structures, and local detail features. The dynamic weight optimization strategy significantly improves robustness to initial point selection errors and image noise, providing a reliable data foundation for subsequent long-term thermal state change analysis and corrosion propagation assessment.

[0100] It is understood that the above specific description of the present invention is only for illustrating the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention to achieve the same technical effect; as long as the use needs are met, they are all within the protection scope of the present invention.

Claims

1. A method for registering high-noise infrared thermal image sequences with dynamic weight optimization, characterized in that... Includes the following steps: Step 1: Obtain at least two infrared sequences of the same target object from different periods as the dataset to be registered and perform preprocessing; Step 2: Using at least two infrared images to be registered obtained in Step 1, one infrared image is used as the reference image and the other infrared images are used as the images to be registered; interactively select the corresponding control point pairs on the images, with each pair of control points corresponding to the same physical feature positions in the reference image and the images to be registered. Step 3: When the number of selected control point pairs is not less than four, the perspective transformation matrix H is calculated using a rigid registration algorithm based on the projection model. The image at one time moment is projected and transformed into the coordinate system of the image at another time moment to achieve initial alignment. With the help of the initially selected control point set, the perspective transformation matrix is ​​solved by the weighted least squares method. The currently registered image is superimposed and displayed in real time on the human-computer interaction interface. A quantitative feedback index of the registration error is given. Step 4: Dynamic weight optimization stage, dynamically iteratively adjusting the weights of the control point pairs; Step 4, the "dynamic weight optimization stage", is connected to Step 2, the "interactive point selection", and Step 3, the "initial H solution"; after generating control points in Step 2 and obtaining the initial H in Step 3, Step 4 is entered to dynamically optimize the weights. The initial weights in Step 4 come from Step 2 and 3; the weights in the point selection stage are calculated according to the process in Step 2 and 3. Step 2 and 3 are the initial stages, and Step 4 is the iterative stage, assigning an initial weight to each pair of control points; Step 5: In multiple iterations, continuously execute the loop of "calculate transformation - residual evaluation - update weights", set a convergence decay factor to gradually reduce the proportion of the initial weights, and make the data-driven weights gradually dominate as the iteration progresses. Step 6: After the weight iterative optimization reaches the convergence condition, determine the final perspective transformation matrix and the weights of each control point, apply the registration transformation to the image in real time, and output the transformation matrix; In step 5, the initial artificial weights are: The default value is K0=8; that is, the first 8 iterations gradually transition from prior knowledge to data-driven decision-making. In step 5, a keypoint weight freeze protection mechanism is introduced: Set a lower limit for high-confidence control points: When its residual satisfies T=3 consecutive times The freeze will be lifted and the system will return to regular updates.

2. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... In step 1, the infrared sequence image obtained at an earlier time is used as the reference image, and the infrared sequence image obtained at a later time is used as the image to be registered.

3. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... In step 1, the preprocessing uses Fast Fourier Transform bandpass denoising to reduce noise in the thermal image sequence and reduce the impact of environmental noise; and the principal component analysis method is used to reduce the dimensionality of the sequence, extracting the main features of the three-dimensional time series data and mapping them onto the two-dimensional image.

4. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 3, characterized in that... The main features are: performing PCA on the time series, taking the top K principal components with a cumulative variance contribution rate ≥80%, and using K=1 as the feature image by default; if <80%, taking K=2 and performing weighted synthesis.

5. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... After obtaining the initial perspective transformation, the reprojection error of each control point is calculated; control points with smaller errors are given higher weights, while those with larger errors have their weights reduced exponentially.

6. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... For the initial weights automatically assigned by the system during the iteration in step 4: Initial weights: Where C i For Harris corner / edge response, V i For the 11×11 neighborhood texture variance, T i This represents the standard deviation of the pixel's temperature over time; the default values ​​are β1=0.5, β2=0.3, β3=0.2, and w... i (0) Normalize to [0.1, 1.0]; Dynamically updated: The default α=1.0, and the residual MAD×1.4826 is used. min =0.05,w max =1.

2.

7. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... During the interaction and iteration process, a cubic interpolation algorithm is used to quickly redraw the registration result frame.

8. The high-noise infrared thermal image sequence registration method with dynamic weight optimization according to claim 1, characterized in that... Step 5 includes an automatic error correction mechanism. During the registration process, the mechanism automatically identifies and corrects erroneous control point pairs through reprojection error detection and control point optimization algorithms.

Images