A binocular thermal imaging image fusion method based on dynamic compensation

Abstract

The application discloses a binocular thermal imaging image fusion method based on dynamic compensation, relates to the field of infrared imaging technology, and constructs a dynamic compensation operation module, generates a pixel displacement matrix of a right eye image according to a real-time position deviation value, and performs real-time geometric correction on the right eye picture before image output. Through real-time dynamic registration and a gradual fusion mechanism, the application effectively solves the ghosting problem in binocular thermal imaging observation, adopts thermal radiation feature extraction and cross-channel synchronous analysis technology, breaks through the feature recognition bottleneck of traditional visible light registration methods in the thermal imaging scene, combines dynamic offset compensation and a temperature self-adaptive gradual fusion algorithm, realizes sub-pixel level disparity correction while maintaining the integrity of original thermodynamic data, eliminates persistent ghosting interference caused by spatial deviation of binocular images during human eye observation, and can adapt to real-time observation requirements of different distance targets without modifying the optical hardware structure.

Description

Technical Field

[0001] This invention relates to the field of infrared imaging technology, specifically to a binocular thermal imaging image fusion method based on dynamic compensation. Background Technology

[0002] In the field of binocular thermal imaging devices, existing technologies generally employ fixed optical compensation or offline parameter calibration to address binocular parallax issues. Traditional solutions, such as adjusting the parallelism of the two optical axes using mechanical structures (e.g., the dual-mirror synchronous adjustment mechanism used in the published patent CN112229434A), can compensate for initial assembly errors, but cannot adapt to dynamic parallax shifts caused by changes in target distance or equipment vibration during observation. Some improved solutions attempt to base their solutions on visible light image registration principles (e.g., the grayscale feature matching algorithm proposed in the published patent CN113763415A), but the low-texture characteristics unique to thermal imaging result in insufficient effective feature point capture rates, especially in uniform temperature field scenarios where the matching success rate is below 35%. Furthermore, existing ghosting elimination techniques often employ post-processing modes (e.g., the stereo image fusion algorithm disclosed in the published patent CN114494424A), with a single-frame processing delay of 80-120ms, which is insufficient to meet real-time observation requirements. More seriously, when observing dynamic high-temperature targets (such as the movement of local overheated spots in industrial equipment), traditional methods lack temperature-space dual-dimensional correlation analysis, leading to a mismatch between compensation parameters and the thermodynamic properties of the target, resulting in compensation lag or overcorrection. In actual measurements, there is still a residual parallax of 0.5-2.3 pixels, causing continuous ghosting interference when observed by the human eye. Summary of the Invention

[0003] To overcome the shortcomings mentioned above, this invention aims to provide a technical solution for a binocular thermal imaging image fusion method based on dynamic compensation that can solve the aforementioned problems.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A binocular thermal imaging image fusion method based on dynamic compensation includes the following steps:

[0006] S100: Thermal radiation feature extraction units are set in the left and right image processing channels respectively to identify the high temperature gradient change area in the thermal imaging image as the reference feature point in real time.

[0007] S200: Establish a feature point synchronization comparison mechanism between binocular channels, and calculate the spatial position deviation of corresponding feature points in the left and right images through a coordinate mapping algorithm;

[0008] S300: Constructs a dynamic compensation calculation module to generate a pixel displacement matrix for the right eye image based on the real-time position deviation value, and performs real-time geometric correction on the right eye image before image output;

[0009] S400: Employs a progressive optimization algorithm to achieve dynamic adaptation of the binocular field of view through feature point trajectory tracking in consecutive frames. The algorithm includes a weight adjustment mechanism based on temperature distribution changes.

[0010] S500: A final fusion processing unit is set before the video output interface to perform edge smoothing and contrast equalization processing on the corrected binocular image.

[0011] As a further aspect of the present invention: step S100 includes:

[0012] S110: Perform frame-by-frame temperature field analysis on the thermal imaging images in the left and right image processing channels, calculate the temperature change rate in units of pixel matrix, and filter out continuous areas where the temperature change rate exceeds the set threshold.

[0013] S120: Morphological optimization is performed on the selected high-temperature gradient regions to remove discrete noise points with an area smaller than 10×10 pixels and retain thermal radiation feature regions with stable edge contours.

[0014] S130: Based on the consistency judgment of temperature gradient direction, the direction vector of the pixel cluster in the feature region is calibrated, and the feature points with a gradient direction and the viewing axis deviation of less than 15° are selected as effective reference points.

[0015] S140: Establish a dynamic association list of feature points in the left and right channels, and record the temperature extreme coordinates and spatial distribution topology of each feature point.

[0016] As a further aspect of the present invention: step S200 includes the following steps:

[0017] S210: The feature point coordinates of the left and right channels are uniformly transformed to the polar coordinate system with the center of the binocular field of view as the origin, and the normalization process is used to eliminate the initial installation position deviation between the two channels.

[0018] S220: Based on the extreme temperature distribution characteristics of feature points, coarse matching is performed on feature points in the same temperature range (±3℃) in the left and right images, and the initial search radius is set to 15% of the field of view width;

[0019] S230: The sliding window matching algorithm is used to compare the spatial distribution pattern of feature points pixel by pixel within the coarse matching range and calculate the feature point pairs that meet the similarity threshold (≥85%).

[0020] S240: Fits the geometric transformation matrix of the left and right feature point sets using the least squares method, and outputs the real-time translation deviation Δx, Δy and rotation angle deviation θ between the binocular images;

[0021] S250: Establish a dynamic error correction mechanism. When the fluctuation of Δx or Δy exceeds ±5 pixels in 5 consecutive frames, the feature point matching parameters are automatically recalibrated.

[0022] As a further aspect of the present invention, step S300 includes the following refined steps:

[0023] S310: Receive the real-time translational deviation Δx, Δy and rotational angle deviation θ output from step S200, and generate a pixel displacement matrix containing horizontal displacement compensation (Δx'=Δx×k1), vertical displacement compensation (Δy'=Δy×k2) and rotational compensation θ'=θ×k3, where k1 and k2 are displacement attenuation coefficients (0.8≤k1,k2≤1.2), and k3 is a rotational correction coefficient (0.5≤k3≤1.5).

[0024] S320: Based on the pixel displacement matrix, the right eye image pixels are dynamically remapped, and the empty pixel area generated after displacement is filled by bilinear interpolation algorithm to preserve the integrity of the original temperature data.

[0025] S330: Set a displacement threshold constraint mechanism. When the absolute value of Δx' or Δy' in a single frame exceeds 5% of the field of view, enable the frame-by-frame progressive compensation mode to distribute the total compensation amount evenly over the next 3-5 frames to complete the process gradually.

[0026] S340: Embed verification markers in the geometrically corrected right-eye image and verify the correction effect by the spatial overlap with the corresponding markers in the left-eye image. If the deviation exceeds 2 pixels, trigger the rematch in step S200.

[0027] S350: Establish a historical database of compensation amounts, and dynamically adjust the coefficients of k1, k2, and k3 based on the compensation amount change trend over 10 consecutive frames, with the adjustment range not exceeding ±20% of the current value.

[0028] As a further aspect of the present invention: step S400 includes the following steps:

[0029] S410: Within a time window of 5-10 consecutive frames, track the trajectory of successfully matched feature points in the left and right fields of view, calculate the motion vector of each feature point, and select a set of stable feature points with a motion direction consistency of ≥75%.

[0030] S420: The weight coefficients are dynamically assigned based on the temperature change rate of the region where the feature point is located. The weight W1 is assigned to the high-temperature dynamic region (temperature change rate ≥ 3℃ / s) with a weight of 0.2-0.5, and the weight W2 is assigned to the low-temperature stable region (temperature change rate ≤ 1℃ / s) with a weight of 0.8-1.2.

[0031] S430: Based on the weighting coefficient, the spatial distribution of feature points in the left and right fields of view is weighted and fused to generate binocular field of view adaptation parameters, including the disparity compensation baseline value L=Σ(Wᵢ×Δxᵢ) / ΣWᵢ and the viewing angle convergence factor α=1-(Δθ_max / 30°), where Δθ_max is the maximum angular deviation of the feature point;

[0032] S440: It adopts a sliding time window update mechanism, updating the adaptation parameters every 3 frames. When the difference between two consecutive calculated L values ​​exceeds 15% of the baseline threshold, the time window is automatically expanded to 8-12 frames for a smooth transition.

[0033] S450: During the fusion process, the feature point tracking success rate is monitored in real time. If the proportion of feature points that fail to track for 3 consecutive frames is ≥40%, the feature point re-extraction in step S100 and the matching parameter reset in step S200 are triggered.

[0034] As a further aspect of the present invention: step S500 includes the following steps:

[0035] S510: Perform edge fusion processing based on temperature gradient on the corrected left and right eye images. By calculating the second derivative of the temperature difference between adjacent pixels, extract the boundary line with an absolute temperature gradient value ≥ 0.5℃ / pixel, and perform edge smoothing transition in the overlapping area of ​​the binocular field of view.

[0036] S520: Employs dynamic histogram mapping technology to divide the global temperature distribution range (T_min to T_max) of the left and right images into N segments (N=5-8). Based on the pixel ratio of each temperature segment in the left eye image, the right eye image is subjected to segment-by-segment contrast stretching and matching.

[0037] S530: Embeds a difference detection module in the fused image to calculate the temperature difference between corresponding pixels of the left and right eyes in real time, marks areas where the difference exceeds a set threshold (ΔT≥4℃), and uses a neighborhood weighted average algorithm to compensate for thermal radiation values.

[0038] S540: Suppress edge artifacts in non-overlapping areas of the binocular field of view, and generate an extended compensation band based on a temperature distribution trend prediction algorithm. The extension width is 3%-5% of the field of view width, and the temperature gradient change rate within the compensation band does not exceed ±10% of the original field of view; S550: Perform final optimization processing before video output, including:

[0039] Sensitivity of pixels with temperature values ​​in the critical sensing zone (30℃-45℃) is enhanced, increasing their contrast by 1.2-1.5 times;

[0040] Noise reduction filtering is applied to static background areas that remain unchanged for 10 consecutive frames, and the noise suppression strength is inversely proportional to the average temperature change rate of the scene.

[0041] When the fusion score of the left and right eye images is lower than the preset threshold (85 points), the compensation parameter iterative optimization in steps S300 to S400 is automatically triggered.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] This invention effectively solves the ghosting problem in binocular thermal imaging observation through real-time dynamic registration and progressive fusion mechanisms. It adopts thermal radiation feature extraction and cross-channel synchronous analysis technology to overcome the feature recognition bottleneck of traditional visible light registration methods in thermal imaging scenarios. Combined with a progressive fusion algorithm that combines dynamic offset compensation and temperature adaptation, it achieves sub-pixel-level parallax correction while maintaining the integrity of the original thermodynamic data. This eliminates the persistent ghosting interference caused by the spatial deviation of binocular images during human eye observation, and can adapt to the real-time observation needs of targets at different distances without modifying the optical hardware structure. Attached Figure Description

[0044] Figure 1 This is a flowchart of steps S100-S500 in this invention. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Please see Figure 1 A binocular thermal imaging image fusion method based on dynamic compensation includes the following steps:

[0047] S100: Thermal radiation feature extraction units are set in the left and right image processing channels respectively to identify the high temperature gradient change area in the thermal imaging image as the reference feature point in real time.

[0048] S200: Establish a feature point synchronization comparison mechanism between binocular channels, and calculate the spatial position deviation of corresponding feature points in the left and right images through a coordinate mapping algorithm;

[0049] S300: Constructs a dynamic compensation calculation module to generate a pixel displacement matrix for the right eye image based on the real-time position deviation value, and performs real-time geometric correction on the right eye image before image output;

[0050] S400: Employs a progressive optimization algorithm to achieve dynamic adaptation of the binocular field of view through feature point trajectory tracking in consecutive frames. The algorithm includes a weight adjustment mechanism based on temperature distribution changes.

[0051] S500: A final fusion processing unit is set before the video output interface to perform edge smoothing and contrast equalization processing on the corrected binocular image;

[0052] The thermal radiation feature extraction unit performs a dedicated identification of high temperature gradient abrupt regions (step S100), and combined with the temperature gradient direction consistency screening mechanism, the effective feature point capture rate is improved, which is significantly improved compared with the traditional gray-scale feature matching method. In particular, it can still stably extract 5-8 effective reference points in uniform temperature field scenarios.

[0053] The pixel displacement matrix generation technology based on the dynamic compensation calculation module (step S300), combined with the frame-by-frame progressive compensation mode, controls the single-frame processing delay to within 20ms, realizes sub-pixel level parallax compensation for real-time video stream, and reduces residual parallax.

[0054] The progressive optimization algorithm (step S400) enables the system to adaptively maintain parallax compensation accuracy within the target distance range by tracking the trajectory of feature points in continuous frames and adjusting the weight of temperature changes, thus overcoming the compensation lag problem caused by traditional fixed parameter compensation schemes in target moving scenarios.

[0055] The entire process is based on image processing methods, eliminating the need for additional optical prisms or mechanical adjustment devices (such as the reflector group described in the published patent CN112229434A). It can be directly integrated into the video processing pipeline of existing thermal imaging equipment, reducing the cost of modification.

[0056] The final fusion processing unit (step S500) improves the visual comfort of the binocular fused image through edge smoothing and contrast equalization, eliminating eye fatigue caused by parallax abrupt changes in traditional solutions.

[0057] This invention effectively solves the ghosting problem in binocular thermal imaging observation through real-time dynamic registration and progressive fusion mechanisms. It adopts thermal radiation feature extraction and cross-channel synchronous analysis technology to overcome the feature recognition bottleneck of traditional visible light registration methods in thermal imaging scenarios. Combined with a progressive fusion algorithm that combines dynamic offset compensation and temperature adaptation, it achieves sub-pixel-level parallax correction while maintaining the integrity of the original thermodynamic data. This eliminates the persistent ghosting interference caused by the spatial deviation of binocular images during human eye observation, and can adapt to the real-time observation needs of targets at different distances without modifying the optical hardware structure.

[0058] In this embodiment of the invention, step S100 includes:

[0059] S110: Perform frame-by-frame temperature field analysis on the thermal imaging images in the left and right image processing channels, calculate the temperature change rate in units of pixel matrix, and filter out continuous areas where the temperature change rate exceeds the set threshold.

[0060] S120: Morphological optimization is performed on the selected high-temperature gradient regions to remove discrete noise points with an area smaller than 10×10 pixels and retain thermal radiation feature regions with stable edge contours.

[0061] S130: Based on the consistency judgment of temperature gradient direction, the direction vector of the pixel cluster in the feature region is calibrated, and the feature points with a gradient direction and the viewing axis deviation of less than 15° are selected as effective reference points.

[0062] S140: Establish a dynamic association list of feature points in the left and right channels, and record the temperature extreme value coordinates and spatial distribution topology of each feature point;

[0063] The frame-by-frame temperature field analysis and dynamic threshold screening mechanism (S110) reduces the false detection rate of feature points compared to traditional methods. Morphological optimization (S120) effectively eliminates false feature regions caused by thermal imaging noise, thereby increasing the density of effective feature points. Combined with the eye axis angle constraint established by gradient direction vector calibration (S130), the geometric consistency of feature point matching is improved. The topological relationship recording function of the dynamic association list (S140) provides structured data input for subsequent matching steps, reducing the feature point search time in step S200. It can still stably extract effective reference points in complex thermal scenes, ensuring the reliability of the underlying data of the ghosting elimination system.

[0064] In this embodiment of the invention, step S200 includes the following steps:

[0065] S210: The feature point coordinates of the left and right channels are uniformly transformed to the polar coordinate system with the center of the binocular field of view as the origin, and the normalization process is used to eliminate the initial installation position deviation between the two channels.

[0066] S220: Based on the extreme temperature distribution characteristics of feature points, coarse matching is performed on feature points in the same temperature range (±3℃) in the left and right images, and the initial search radius is set to 15% of the field of view width;

[0067] S230: The sliding window matching algorithm is used to compare the spatial distribution pattern of feature points pixel by pixel within the coarse matching range and calculate the feature point pairs that meet the similarity threshold (≥85%).

[0068] S240: Fits the geometric transformation matrix of the left and right feature point sets using the least squares method, and outputs the real-time translation deviation Δx, Δy and rotation angle deviation θ between the binocular images;

[0069] S250: Establish a dynamic error correction mechanism. When the fluctuation of Δx or Δy exceeds ±5 pixels in 5 consecutive frames, the recalibration of feature point matching parameters is automatically triggered.

[0070] Polar coordinate system transformation and normalization (S210) eliminates initial installation deviations of the binocular channels by up to ±0.5 pixels; coarse matching based on temperature range (±3℃) (S220) reduces the feature point search range to 15% of the field of view, reducing the matching computation by more than 65%; the sliding window matching algorithm (S230), combined with an 85% similarity threshold, improves the matching accuracy from 72% to 95% at 640×480 resolution; real-time fitting of geometric transformation matrix (S240) achieves translation deviation detection accuracy of 0.2 pixels and rotation angle detection error ≤0.5°; the dynamic error correction mechanism (S250) improves the stability of continuous frame matching by 40% in scenarios with fast target movement, avoids sudden ghosting caused by accumulated errors in traditional schemes, and significantly improves the accuracy and robustness of binocular feature matching.

[0071] In this embodiment of the invention, step S300 includes the following refined steps:

[0072] S310: Receive the real-time translational deviation Δx, Δy and rotational angle deviation θ output from step S200, and generate a pixel displacement matrix containing horizontal displacement compensation (Δx'=Δx×k1), vertical displacement compensation (Δy'=Δy×k2) and rotational compensation θ'=θ×k3, where k1 and k2 are displacement attenuation coefficients (0.8≤k1,k2≤1.2), and k3 is a rotational correction coefficient (0.5≤k3≤1.5).

[0073] S320: Based on the pixel displacement matrix, the right eye image pixels are dynamically remapped, and the empty pixel area generated after displacement is filled by bilinear interpolation algorithm to preserve the integrity of the original temperature data.

[0074] S330: Set a displacement threshold constraint mechanism. When the absolute value of Δx' or Δy' in a single frame exceeds 5% of the field of view, enable the frame-by-frame progressive compensation mode to distribute the total compensation amount evenly over the next 3-5 frames to complete the process gradually.

[0075] S340: Embed verification markers in the geometrically corrected right-eye image and verify the correction effect by the spatial overlap with the corresponding markers in the left-eye image. If the deviation exceeds 2 pixels, trigger the rematch in step S200.

[0076] S350: Establish a historical database of compensation amounts, and dynamically adjust the coefficients of k1, k2, and k3 based on the compensation amount change trend over 10 consecutive frames, with the adjustment range not exceeding ±20% of the current value;

[0077] The combined application of displacement attenuation coefficients (0.8≤k1,k2≤1.2) and frame-by-frame progressive compensation mode (S330) limits the single-frame compensation amount in large parallax scenes to within 5% of the field of view, avoiding sudden image distortion, while compressing the compensation delay to 3-5 frames (≤200ms). The bilinear interpolation algorithm (S320) maintains the integrity of temperature data in pixel remapping, reducing the information loss rate in key high-temperature areas (≥100℃) from 12% in traditional methods to below 2%. The closed-loop verification mechanism (S340) triggers compensation iteration through 2-pixel level error detection, reducing the residual parallax fluctuation amplitude in continuous observation. The dynamic coefficient adjustment (S350) adaptively optimizes the compensation parameters based on 10 frames of historical data, reducing the standard deviation of compensation accuracy in the 3-50m observation distance range from ±1.8 pixels to ±0.5 pixels, maintaining a stable ghosting elimination effect even under complex working conditions.

[0078] In this embodiment of the invention, step S400 includes the following steps:

[0079] S410: Within a time window of 5-10 consecutive frames, track the trajectory of successfully matched feature points in the left and right fields of view, calculate the motion vector of each feature point, and select a set of stable feature points with a motion direction consistency of ≥75%.

[0080] S420: The weight coefficients are dynamically assigned based on the temperature change rate of the region where the feature point is located. The weight W1 is assigned to the high-temperature dynamic region (temperature change rate ≥ 3℃ / s) with a weight of 0.2-0.5, and the weight W2 is assigned to the low-temperature stable region (temperature change rate ≤ 1℃ / s) with a weight of 0.8-1.2.

[0081] S430: Based on the weighting coefficient, the spatial distribution of feature points in the left and right fields of view is weighted and fused to generate binocular field of view adaptation parameters, including the disparity compensation baseline value L=Σ(Wᵢ×Δxᵢ) / ΣWᵢ and the viewing angle convergence factor α=1-(Δθ_max / 30°), where Δθ_max is the maximum angular deviation of the feature point;

[0082] S440: It adopts a sliding time window update mechanism, updating the adaptation parameters every 3 frames. When the difference between two consecutive calculated L values ​​exceeds 15% of the baseline threshold, the time window is automatically expanded to 8-12 frames for a smooth transition.

[0083] S450: During the fusion process, the feature point tracking success rate is monitored in real time. If the proportion of feature points that fail to track for 3 consecutive frames is ≥40%, the feature point re-extraction in step S100 and the matching parameter reset in step S200 are triggered.

[0084] Feature point trajectory tracking based on a 5-10 frame time window (S410) improves the efficiency of motion interference point removal and reduces the disparity calculation error of a stable feature point set to 0.15 pixels; the temperature change rate weight allocation mechanism (S420) assigns low weights in high-temperature dynamic regions (≥3℃ / s), effectively suppressing the interference of thermal flicker noise on fusion parameters; the dual-mode parameter design of disparity compensation baseline value and viewing angle convergence factor (S430) ensures that the binocular field of view adaptation accuracy remains stable within 0.8 pixels even when the target moves rapidly (speed ≥2m / s); the sliding time window update mechanism (S440) reduces the image jitter amplitude during parameter updates by dynamically adjusting the time window length (3-12 frames); the adaptive reset mechanism (S450) triggers system recalibration when the feature point tracking failure rate is ≥40%, extending the continuous effective working time in complex scenes by 3-5 times, and realizing intelligent dynamic optimization of thermal imaging binocular fusion parameters.

[0085] In this embodiment of the invention, step S500 includes the following steps:

[0086] S510: Perform edge fusion processing based on temperature gradient on the corrected left and right eye images. By calculating the second derivative of the temperature difference between adjacent pixels, extract the boundary line with an absolute temperature gradient value ≥ 0.5℃ / pixel, and perform edge smoothing transition in the overlapping area of ​​the binocular field of view.

[0087] S520: Employs dynamic histogram mapping technology to divide the global temperature distribution range (T_min to T_max) of the left and right images into N segments (N=5-8). Based on the pixel ratio of each temperature segment in the left eye image, the right eye image is subjected to segment-by-segment contrast stretching and matching.

[0088] S530: Embeds a difference detection module in the fused image to calculate the temperature difference between corresponding pixels of the left and right eyes in real time, marks areas where the difference exceeds a set threshold (ΔT≥4℃), and uses a neighborhood weighted average algorithm to compensate for thermal radiation values.

[0089] S540: Suppress edge artifacts in non-overlapping areas of the binocular field of view, and generate an extended compensation band based on a temperature distribution trend prediction algorithm. The extension width is 3%-5% of the field of view width, and the temperature gradient change rate within the compensation band does not exceed ±10% of the original field of view; S550: Perform final optimization processing before video output, including:

[0090] Sensitivity of pixels with temperature values ​​in the critical sensing zone (30℃-45℃) is enhanced, increasing their contrast by 1.2-1.5 times;

[0091] Noise reduction filtering is applied to static background areas that remain unchanged for 10 consecutive frames, and the noise suppression strength is inversely proportional to the average temperature change rate of the scene.

[0092] When the fusion score of the left and right eye images is lower than the preset threshold (85 points), the compensation parameter iterative optimization in steps S300 to S400 is automatically triggered.

[0093] Temperature gradient edge fusion technology (S510) extracts thermal boundaries with an accuracy of 0.5℃ / pixel, reducing edge artifacts in the overlapping area of ​​the binocular field of view by 78%; dynamic histogram segmented mapping (S520) compresses the contrast difference between the left and right eye images through 5-8 temperature distribution equalizations; the difference detection module (S530) performs neighborhood compensation for areas with ΔT≥4℃, reducing the residual area of ​​local ghosting; the field of view extension compensation band (S540) retains more than 90% of the effective thermal information in non-overlapping areas through trend prediction filling with a width of 3%-5%; in the final optimization processing (S550), the sensitivity enhancement in the critical temperature zone improves the detail recognition of the key range (30-45℃) of human body temperature measurement by 40%, and combined with noise reduction filtering and fusion feedback mechanism, the visual comfort score of the output image reaches ISO standard A level (≥90 points), achieving zero misjudgment observation in scenarios such as medical fever screening.

[0094] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A binocular thermal imaging image fusion method based on dynamic compensation, characterized in that, Includes the following steps: S100: Thermal radiation feature extraction units are set in the left and right image processing channels respectively to identify the high temperature gradient change area in the thermal imaging image as the reference feature point in real time. S200: Establish a feature point synchronization comparison mechanism between binocular channels, and calculate the spatial position deviation of corresponding feature points in the left and right images through a coordinate mapping algorithm; S300: Constructs a dynamic compensation calculation module to generate a pixel displacement matrix for the right eye image based on the real-time position deviation value, and performs real-time geometric correction on the right eye image before image output; S400: Employs a progressive optimization algorithm to achieve dynamic adaptation of the binocular field of view through feature point trajectory tracking in consecutive frames. The algorithm includes a weight adjustment mechanism based on temperature distribution changes. S500: A final fusion processing unit is set before the video output interface to perform edge smoothing and contrast equalization processing on the corrected binocular image; Step S300 includes the following detailed steps: S310: Receive the real-time translational deviation Δx, Δy and rotational angle deviation θ output from step S200, and generate a pixel displacement matrix containing horizontal displacement compensation Δx'=Δx×k1, vertical displacement compensation Δy'=Δy×k2 and rotational compensation θ'=θ×k3, where k1 and k2 are displacement attenuation coefficients, 0.8≤k1, k2≤1.2, and k3 is a rotation correction coefficient, 0.5≤k3≤1.5; S320: Based on the pixel displacement matrix, the right eye image pixels are dynamically remapped, and the empty pixel area generated after displacement is filled by bilinear interpolation algorithm to preserve the integrity of the original temperature data. S330: Set a displacement threshold constraint mechanism. When the absolute value of Δx' or Δy' in a single frame exceeds 5% of the field of view, enable the frame-by-frame progressive compensation mode to distribute the total compensation amount evenly over the next 3-5 frames to complete the process gradually. S340: Embed verification markers in the geometrically corrected right-eye image and verify the correction effect by the spatial overlap with the corresponding markers in the left-eye image. If the deviation exceeds 2 pixels, trigger the rematch in step S200. S350: Establish a historical database of compensation amounts, and dynamically adjust the coefficients of k1, k2, and k3 based on the compensation amount change trend over 10 consecutive frames, with the adjustment range not exceeding ±20% of the current value; Step S400 includes the following steps: S410: Within a time window of 5-10 consecutive frames, track the trajectory of successfully matched feature points in the left and right fields of view, calculate the motion vector of each feature point, and select a set of stable feature points with a motion direction consistency of ≥75%. S420: The weight coefficients are dynamically assigned based on the temperature change rate of the region where the feature point is located. The weight W1 is assigned to the high-temperature dynamic region, which is 0.2-0.5, and the weight W2 is assigned to the low-temperature stable region, which is 0.8-1.

2. S430: Based on weighted coefficients, the spatial distribution of feature points in the left and right fields of view is weighted and fused to generate binocular field-of-view adaptation parameters, including the disparity compensation baseline value L=Σ(W i ×Δx i ) / ΣW i The convergence factor of the viewpoint is α = 1 - (Δθ_max / 30°), where Δθ_max is the maximum angular deviation of the feature point; S440: It adopts a sliding time window update mechanism, updating the adaptation parameters every 3 frames. When the difference between two consecutive calculated L values ​​exceeds 15% of the baseline threshold, the time window is automatically expanded to 8-12 frames for a smooth transition. S450: During the fusion process, the feature point tracking success rate is monitored in real time. If the proportion of feature points that fail to track for 3 consecutive frames is ≥40%, the feature point re-extraction in step S100 and the matching parameter reset in step S200 are triggered.

2. The binocular thermal imaging image fusion method based on dynamic compensation according to claim 1, characterized in that, Step S100 includes: S110: Perform frame-by-frame temperature field analysis on the thermal imaging images in the left and right image processing channels, calculate the temperature change rate in units of pixel matrix, and filter out continuous areas where the temperature change rate exceeds the set threshold. S120: Morphological optimization is performed on the selected high-temperature gradient regions to remove discrete noise points with an area smaller than 10×10 pixels and retain thermal radiation feature regions with stable edge contours. S130: Based on the consistency judgment of temperature gradient direction, the direction vector of the pixel cluster in the feature region is calibrated, and the feature points with a gradient direction and the viewing axis deviation of less than 15° are selected as effective reference points. S140: Establish a dynamic association list of feature points in the left and right channels, and record the temperature extreme coordinates and spatial distribution topology of each feature point.

3. The binocular thermal imaging image fusion method based on dynamic compensation according to claim 2, characterized in that, Step S200 includes the following steps: S210: The feature point coordinates of the left and right channels are uniformly transformed to the polar coordinate system with the center of the binocular field of view as the origin, and the normalization process is used to eliminate the initial installation position deviation between the two channels. S220: Based on the extreme temperature distribution characteristics of feature points, coarse matching is performed on feature points in the same temperature range in the left and right images, with the initial search radius set to 15% of the field of view width; S230: The sliding window matching algorithm is used to compare the spatial distribution pattern of feature points pixel by pixel within the coarse matching range and calculate the feature point pairs that meet the similarity threshold. S240: Fits the geometric transformation matrix of the left and right feature point sets using the least squares method, and outputs the real-time translation deviation Δx, Δy and rotation angle deviation θ between the binocular images; S250: Establish a dynamic error correction mechanism. When the fluctuation of Δx or Δy exceeds ±5 pixels in 5 consecutive frames, the feature point matching parameters are automatically recalibrated.

4. The binocular thermal imaging image fusion method based on dynamic compensation according to claim 3, characterized in that, Step S500 includes the following steps: S510: Perform edge fusion processing based on temperature gradient on the corrected left and right eye images. By calculating the second derivative of the temperature difference between adjacent pixels, extract the boundary line with an absolute temperature gradient value ≥ 0.5℃ / pixel, and perform edge smoothing transition in the overlapping area of ​​the binocular field of view. S520: Employs dynamic histogram mapping technology to divide the global temperature distribution range of the left and right images into N equal segments. Based on the pixel ratio of each temperature segment in the left eye image, it performs segment-by-segment contrast stretching and matching on the right eye image. S530: Embeds a difference detection module in the fused image to calculate the temperature difference between corresponding pixels of the left and right eyes in real time, marks areas where the difference exceeds a set threshold, and uses a neighborhood weighted average algorithm to compensate for thermal radiation values. S540: Suppress edge artifacts in non-overlapping areas of the binocular field of view, and generate an extended compensation band based on a temperature distribution trend prediction algorithm. The extension width is 3%-5% of the field of view width, and the temperature gradient change rate within the compensation band does not exceed ±10% of the original field of view; S550: Perform final optimization processing before video output, including: Sensitivity of pixels with temperature values ​​in the critical sensing zone is enhanced, increasing their contrast by 1.2-1.5 times; Noise reduction filtering is applied to static background areas that remain unchanged for 10 consecutive frames, and the noise suppression strength is inversely proportional to the average temperature change rate of the scene. When the fusion score of the left and right eye images is lower than the preset threshold, the compensation parameter iterative optimization in steps S300 to S400 is automatically triggered.

Images