High-precision image analysis system and method based on infrared thermal imaging

Abstract

The invention relates to the field of image analysis, in particular to a high-precision image analysis system and method based on infrared thermal imaging, and the system comprises an instrument imaging module, a background filtering module, a target recognition module, an image fusion module and a distortion correction module. The background filtering module is used for separating background noise, the target recognition module is used for selecting feature points and determining target tracks of the feature points, the image fusion module is used for fusing and aligning an infrared image and a visible light image, and the distortion correction module is used for correcting nonlinear geometric distortion in the infrared image. The detection and identification accuracy can be improved, the contour and characteristics of the infrared image are clearer, the target shooting effect is improved, meanwhile, the geometric appearance of the image is restored, the requirement of high-precision application is met, and the overall performance of an imaging system is improved.

Description

Technical Field

[0001] The present invention relates to the field of image analysis, and in particular to a high-precision image analysis system and method based on infrared thermal imaging. Background Art

[0002] Infrared thermography uses infrared radiation to detect surface temperature distribution and generate thermal images. This technology captures infrared radiation emitted by an object and converts it into a visible thermal image, revealing surface temperature differences. In infrared images, different temperatures are typically represented by varying grayscale levels, with red and blue often used to identify these grayscale levels, creating a readable infrared image.

[0003] In complex environments, due to factors such as thermal equilibrium evolution, transmission distance, and atmospheric attenuation, infrared images contain a high level of noise and background interference, such as shot noise and thermal noise, which limits the ability to detect small and weak targets. In most use cases of infrared imagers, it is necessary to identify and track small and weak moving targets. Environmental noise limits the accuracy of thermal images, restricting the infrared detector's imaging capabilities and its ability to generate high-precision images.

[0004] Since the camera lens of the infrared detector is a spherical lens, the infrared image is prone to nonlinear geometric distortion. The existing technology corrects the distortion by fusing visible light images. However, due to the differences in optical axes and parameters between the visible light image and the infrared image, there will be misalignment errors in the image, and the effect of repairing the geometric distortion of high-precision infrared images is not good. Summary of the Invention

[0005] The object of the present invention is to provide a high-precision image analysis system and method based on infrared thermal imaging to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a high-precision image analysis system based on infrared thermal imaging, comprising: an instrument imaging module, a background filtering module, a target recognition module, an image fusion module and a distortion correction module;

[0007] The instrument imaging module is used to connect the infrared imaging recognition and visible light imaging devices. After the infrared imaging device continuously shoots a fixed number of frames, the visible light imaging device simultaneously shoots the target, obtains an infrared image stream by the infrared imaging device, records the pixel grayscale of each frame in the infrared image stream, and obtains a visible light image by the visible light imaging device, and records the pixel color in the visible light image;

[0008] The background filtering module is used to use the AI ​​image segmentation model to identify static classifications from adjacent frame images, perform time-domain filtering on the corresponding non-static classification parts in the infrared image stream, use the sum of squared errors of the pixel brightness sequences between adjacent frames as the screening threshold, filter out pixels that do not meet the threshold conditions as noise points, and use the difference between the average brightness of the noise point and the adjacent pixel points as the noise brightness to separate the background noise from the actual signal, and calculate the Gaussian distribution parameters of the background noise;

[0009] The target recognition module is configured to calculate the diffraction spread radius of each pixel in the non-static fractal portion of the infrared image stream, select pixels with a diffraction spread radius greater than a threshold as feature points, simulate trajectories of the feature points in the infrared image stream, calculate the probability distribution and likelihood ratio of each trajectory based on Gaussian distribution parameters of noise, select the trajectory with the maximum likelihood ratio as the target trajectory of the pixel, and mark all feature points on the trajectory as moving points when the target trajectory meets a continuity condition;

[0010] The image fusion module is used to take the moving point as the center and all pixels within the diffraction expansion radius constitute the target area, perform color transformation on the target area, determine the color value of the moving target point, compare it with the color value of each pixel in the corresponding target area of ​​the visible light image, select the pixel point with the closest color value as the corresponding point, and after the infrared image and the visible image are aligned, use the grayscale transformation algorithm to fuse the infrared image and the visible light image;

[0011] The distortion correction module is used to determine the horizontal and vertical pixel offset values ​​of the infrared lens based on the corresponding offsets of the infrared image and the visible image, introduce the bundle method regional block adjustment through vertical feature point correction technology, establish a calibration model, correct the nonlinear geometric distortion in the infrared image, and output the distortion-corrected infrared image.

[0012] Furthermore, the imaging module of the instrument includes: an infrared imaging unit and a visible light imaging unit;

[0013] The infrared imaging unit is used to control the infrared imaging device to continuously capture infrared images of the target and generate an infrared image stream;

[0014] The visible light imaging unit is used to calculate the shooting frame number of the infrared device, and shoot the target simultaneously with the infrared imaging device every fixed frame number to generate a visible light image.

[0015] Furthermore, the background filtering module includes: an image segmentation unit, a time domain filtering unit and a noise separation unit;

[0016] The image segmentation unit is used to segment the visible light image to segment the static fractals in adjacent visible light images;

[0017] The time domain filtering unit is used to correspond the static classification to the infrared image and filter the pixel grayscale in the infrared image;

[0018] The noise separation unit is used to identify noise points according to the time domain filtering result and separate the background noise.

[0019] Furthermore, the target recognition module includes: a feature extraction unit, a trajectory prediction unit and a dynamic recognition unit;

[0020] The feature extraction unit is used to calculate the diffraction expansion radius of each pixel point and determine the feature point of the starting trajectory;

[0021] The trajectory prediction unit is used to simulate the trajectory of the feature points to obtain the motion trajectory with the highest probability of the feature points;

[0022] The dynamic recognition unit is used to determine the motion speed and motion direction of the feature point according to the length of the motion trajectory and the frame duration.

[0023] Furthermore, the image fusion module includes: a region contrast unit and a grayscale conversion unit;

[0024] The regional comparison unit is used to compare the infrared image and the visible light image to determine the corresponding positions between the images;

[0025] The grayscale conversion unit is used to convert the infrared grayscale image into a readable visible light image using a grayscale conversion algorithm.

[0026] Furthermore, the distortion correction module includes: a deviation comparison unit and a geometric restoration unit;

[0027] The deviation comparison unit is used to determine the horizontal and vertical deviations of the infrared images based on the corresponding positions between the images;

[0028] The geometric restoration unit is used to establish a deviation correction model to eliminate nonlinear geometric distortion in infrared images.

[0029] A high-precision image analysis method based on infrared thermal imaging, comprising the following steps:

[0030] Step S1. When the infrared imager captures a preset number of frames, a visible light imaging device with the same resolution is used to simultaneously capture visible light images, and the infrared image and visible light image are stored separately to obtain an infrared image stream and a visible light image stream;

[0031] Step S2: Temporal filtering is performed on the infrared image stream. The sum of squared errors of pixel brightness sequences between adjacent frames is used as the screening threshold. Pixels that do not meet the threshold are filtered out as noise points. The brightness difference of noise points in adjacent frames is used as the noise intensity to separate the background noise.

[0032] Step S3. Calculate the diffraction expansion radius of each pixel in the infrared image. Pixels with radii above a threshold are considered feature points. Simulate the motion trajectory of the feature points by calculating their movement probability within adjacent frames. The trajectory with the maximum likelihood ratio is used as the target trajectory of the pixel. Adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory.

[0033] Step S4. Using the AI ​​model to segment the static fractals and dynamic fractals in the visible light image stream, the target trajectory in the infrared image is color-converted and matched with the dynamic fractals in the visible light image using color as the standard. The matching errors in the horizontal and vertical directions of the trajectory are used as the horizontal and vertical deviations of the infrared image.

[0034] Step S5. Based on the horizontal and vertical deviations of the infrared image, bundle block adjustment is introduced to identify and calibrate the nonlinear geometric distortion in the infrared image through vertical feature correction technology.

[0035] Furthermore, step S1 includes:

[0036] Step S11. Using a fusion device to connect the infrared imaging recognition and visible light imaging devices, the fusion device includes: a front-end detection device, an optical fiber transmission device, a timing chip, and a back-end control device, wherein the timing chip communicates with the image database in real time to count the infrared imaging frames;

[0037] Step S12. Every time the infrared imaging device takes N image frames, the visible light imaging device takes one image frame, where N is a preset frame rate ratio. The infrared image and the visible light image are stored in the image database in chronological order to form an infrared image stream and a visible light image stream.

[0038] Furthermore, step S2 includes:

[0039] Step S21: Perform time domain filtering on each pixel position in the infrared image stream. If the pixel does not meet the threshold condition, the pixel is marked as a noise point. The threshold condition is:

[0040]

[0041] Where m represents the starting frame number, L represents the duration of the noise point, both m and L are constants, and m < N, L ≤ Nm-1, R(k) represents the brightness of the pixel in the kth frame, ft represents the average value of the standard Gaussian distribution function within the frame duration, A represents the maximum brightness of the pixel in all frames, C is the background constant of the infrared device, and T1 is the preset threshold;

[0042] Step S22. Calculate the brightness difference of the noise points in adjacent frames to form a time domain sequence r = [r(1), r(2), ..., r(L-1)], where r(1) to r(L-1) represent the brightness difference between the 1st frame and the 2nd frame, and the brightness difference between the L-1th frame and the Lth frame, respectively. Use the Gaussian distribution function to fit the time domain sequence r to obtain the background noise function f(t) at the pixel point, where t represents time.

[0043] Furthermore, step S3 includes:

[0044] Step S31. Calculate the diffraction expansion radius of the pixel point (x, y). The calculation equation is:

[0045]

[0046] Where R is the brightness of the pixel in the visible light image, A1 is the brightness of the pixel in the infrared image, x and y represent the horizontal and vertical coordinates of the pixel in the infrared image, ε is the diffraction expansion radius of the pixel, i is the preset pixel offset, and e is the base of the natural logarithm;

[0047] Step S32: Pixels with diffraction expansion radii above a threshold are selected as feature points. Using image flow analysis software, pixels with brightness differences from the feature points below a threshold are selected from adjacent frames as trajectory points of the feature points. All trajectory points are concatenated to generate a motion trajectory, and the simulation results of the motion trajectories of all feature points are output.

[0048] Step S33. Judge the simulation results of all trajectories and calculate the target likelihood ratio of the trajectory:

[0049]

[0050] Where L[P(k)] represents the target likelihood ratio of the trajectory, P(k) represents the pixel point of the trajectory in the kth frame, σ 2 is the variance of the Gaussian distribution obeyed by the background noise function f(t), U represents the number of pixels in the trajectory, and I(k) represents the brightness of the trajectory in the kth frame;

[0051] Step S34: Select the trajectory with the largest target likelihood ratio as the target trajectory of the feature point, predict the position of the target trajectory in the next frame of the image, and adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory.

[0052] Furthermore, step S4 includes:

[0053] Step S41: Segmenting the visible light image stream into static fractals and dynamic fractals, wherein the static fractals represent the portion of the visible light image stream where the color does not change, and the dynamic fractals represent the portion where the color changes;

[0054] Step S42. Use a grayscale fusion algorithm or a morphological image fusion algorithm to convert the target trajectory into a color trajectory, determine the position of the trajectory in the dynamic fractal through a color matching algorithm, and record the average matching error of the target trajectory in the horizontal and vertical directions as the horizontal and vertical deviations of the infrared image.

[0055] Furthermore, step S5 includes:

[0056] Step S51: For each frame in the infrared image stream, bundle block adjustment is introduced to calculate the calibration value of the pixel point (x, y) in the infrared image:

[0057]

[0058] Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical deviations, respectively; Dx and Dy represent the maximum values ​​of the pixel coordinates in the horizontal and vertical directions, respectively;

[0059] Step S52: Adjust the pixel positions in the infrared image according to the calculated calibration amount, eliminate the nonlinear geometric distortion in the infrared image, and output the corrected infrared image stream.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] 1. The present invention uses infrared imaging recognition and visible light imaging equipment to simultaneously photograph the imaging target, identify solid-state fractals from visible light images, perform time-domain filtering on the solid-state fractal portion of the infrared image stream, and use the error of the pixel brightness sequence as the screening threshold to separate background noise. This helps to improve the accuracy of detection and recognition, and can significantly enhance the quality of infrared images, making them easier to interpret.

[0062] 2. In the dynamic fractal part of the infrared image stream, the present invention selects pixel points with an expansion radius higher than a threshold as feature points, performs trajectory simulation on the feature points, and selects the trajectory with the maximum likelihood ratio as the target trajectory of the pixel points. When the target trajectory meets the continuity condition, it is marked as a moving target point. When detecting targets with slight temperature differences, the detection accuracy is higher, the contours and features of the infrared image are clearer, and the target shooting effect is improved.

[0063] 3. The present invention takes the moving target point as the center, compares the color value of each pixel in the corresponding target area of ​​the visible light image, selects the pixel point with the closest color value as the corresponding point, and corrects the nonlinear geometric distortion through feature point correction technology, which can repair the geometric appearance of the image, improve the infrared measurement accuracy, meet the needs of high-precision applications, and enhance the overall performance of the imaging system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 This is a structural schematic diagram of a high-precision image analysis system based on infrared thermal imaging according to the present invention;

[0066] Figure 2 This is a schematic diagram of the steps of a high-precision image analysis method based on infrared thermal imaging of the present invention. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0068] See also Figure 1 , the present invention provides a technical solution: a high-precision image analysis system based on infrared thermal imaging, including: an instrument imaging module, a background filtering module, a target recognition module, an image fusion module and a distortion correction module;

[0069] The instrument imaging module is used to connect the infrared imaging recognition and visible light imaging devices. After the infrared imaging device continuously shoots a fixed number of frames, the visible light imaging device simultaneously shoots the target, obtains an infrared image stream by the infrared imaging device, records the pixel grayscale of each frame in the infrared image stream, and obtains a visible light image by the visible light imaging device, and records the pixel color in the visible light image;

[0070] The instrument imaging module includes: an infrared imaging unit and a visible light imaging unit;

[0071] The infrared imaging unit is used to control the infrared imaging device to continuously capture infrared images of the target and generate an infrared image stream;

[0072] The visible light imaging unit is used to calculate the shooting frame number of the infrared device, and shoot the target simultaneously with the infrared imaging device every fixed frame number to generate a visible light image.

[0073] The background filtering module is used to use the AI ​​image segmentation model to identify static classifications from adjacent frame images, perform time-domain filtering on the corresponding non-static classification parts in the infrared image stream, use the sum of squared errors of the pixel brightness sequences between adjacent frames as the screening threshold, filter out pixels that do not meet the threshold conditions as noise points, and use the difference between the average brightness of the noise point and the adjacent pixel points as the noise brightness to separate the background noise from the actual signal, and calculate the Gaussian distribution parameters of the background noise;

[0074] The background filtering module includes: an image segmentation unit, a time domain filtering unit and a noise separation unit;

[0075] The image segmentation unit is used to segment the visible light image to segment the static fractals in adjacent visible light images;

[0076] The time domain filtering unit is used to correspond the static classification to the infrared image and filter the pixel grayscale in the infrared image;

[0077] The noise separation unit is used to identify noise points according to the time domain filtering result and separate the background noise.

[0078] The target recognition module is configured to calculate the diffraction spread radius of each pixel in the non-static fractal portion of the infrared image stream, select pixels with a diffraction spread radius greater than a threshold as feature points, simulate trajectories of the feature points in the infrared image stream, calculate the probability distribution and likelihood ratio of each trajectory based on Gaussian distribution parameters of noise, select the trajectory with the maximum likelihood ratio as the target trajectory of the pixel, and mark all feature points on the trajectory as moving points when the target trajectory meets a continuity condition;

[0079] The target recognition module includes: a feature extraction unit, a trajectory prediction unit and a dynamic recognition unit;

[0080] The feature extraction unit is used to calculate the diffraction expansion radius of each pixel point and determine the feature point of the starting trajectory;

[0081] The trajectory prediction unit is used to simulate the trajectory of the feature points to obtain the motion trajectory with the highest probability of the feature points;

[0082] The dynamic recognition unit is used to determine the motion speed and motion direction of the feature point according to the length of the motion trajectory and the frame duration.

[0083] The image fusion module is used to take the moving point as the center and all pixels within the diffraction expansion radius constitute the target area, perform color transformation on the target area, determine the color value of the moving target point, compare it with the color value of each pixel in the corresponding target area of ​​the visible light image, select the pixel point with the closest color value as the corresponding point, and after the infrared image and the visible image are aligned, use the grayscale transformation algorithm to fuse the infrared image and the visible light image;

[0084] The image fusion module includes: a region contrast unit and a grayscale conversion unit;

[0085] The regional comparison unit is used to compare the infrared image and the visible light image to determine the corresponding positions between the images;

[0086] The grayscale conversion unit is used to convert the infrared grayscale image into a readable visible light image using a grayscale conversion algorithm.

[0087] The distortion correction module is used to determine the horizontal and vertical pixel offset values ​​of the infrared lens based on the corresponding offsets of the infrared image and the visible image, introduce the bundle method regional block adjustment through vertical feature point correction technology, establish a calibration model, correct the nonlinear geometric distortion in the infrared image, and output the distortion-corrected infrared image.

[0088] The distortion correction module includes: a deviation comparison unit and a geometric restoration unit;

[0089] The deviation comparison unit is used to determine the horizontal and vertical deviations of the infrared images based on the corresponding positions between the images;

[0090] The geometric restoration unit is used to establish a deviation correction model to eliminate nonlinear geometric distortion in infrared images.

[0091] like Figure 2 As shown, a high-precision image analysis method based on infrared thermal imaging includes the following steps:

[0092] Step S1. When the infrared imager captures a preset number of frames, a visible light imaging device with the same resolution is used to simultaneously capture visible light images, and the infrared image and visible light image are stored separately to obtain an infrared image stream and a visible light image stream;

[0093] Step S1 includes:

[0094] Step S11. Using a fusion device to connect the infrared imaging recognition and visible light imaging devices, the fusion device includes: a front-end detection device, an optical fiber transmission device, a timing chip, and a back-end control device, wherein the timing chip communicates with the image database in real time to count the infrared imaging frames;

[0095] Step S12. Every time the infrared imaging device takes N image frames, the visible light imaging device takes one image frame, where N is a preset frame rate ratio. The infrared image and the visible light image are stored in the image database in chronological order to form an infrared image stream and a visible light image stream.

[0096] Step S2: Temporal filtering is performed on the infrared image stream. The sum of squared errors of pixel brightness sequences between adjacent frames is used as the screening threshold. Pixels that do not meet the threshold are filtered out as noise points. The brightness difference of noise points in adjacent frames is used as the noise intensity to separate the background noise.

[0097] Step S2 includes:

[0098] Step S21: Perform time domain filtering on each pixel position in the infrared image stream. If the pixel does not meet the threshold condition, the pixel is marked as a noise point. The threshold condition is:

[0099]

[0100] Where m represents the starting frame number, L represents the duration of the noise point, both m and L are constants, and m < N, L ≤ Nm-1, R(k) represents the brightness of the pixel in the kth frame, ft represents the average value of the standard Gaussian distribution function within the frame duration, A represents the maximum brightness of the pixel in all frames, C is the background constant of the infrared device, and T1 is the preset threshold;

[0101] Step S22. Calculate the brightness difference of the noise points in adjacent frames to form a time domain sequence r = [r(1), r(2), ..., r(L-1)], where r(1) to r(L-1) represent the brightness difference between the 1st frame and the 2nd frame, and the brightness difference between the L-1th frame and the Lth frame, respectively. Use the Gaussian distribution function to fit the time domain sequence r to obtain the background noise function f(t) at the pixel point, where t represents time.

[0102] Step S3. Calculate the diffraction expansion radius of each pixel in the infrared image. Pixels with radii above a threshold are considered feature points. Simulate the motion trajectory of the feature points by calculating their movement probability within adjacent frames. The trajectory with the maximum likelihood ratio is used as the target trajectory of the pixel. Adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory.

[0103] Step S3 includes:

[0104] Step S31. Calculate the diffraction expansion radius of the pixel point (x, y). The calculation equation is:

[0105]

[0106] Where R is the brightness of the pixel in the visible light image, A1 is the brightness of the pixel in the infrared image, x and y represent the horizontal and vertical coordinates of the pixel in the infrared image, ε is the diffraction expansion radius of the pixel, i is the preset pixel offset, and e is the base of the natural logarithm;

[0107] Step S32: Pixels with diffraction expansion radii above a threshold are selected as feature points. Using image flow analysis software, pixels with brightness differences from the feature points below a threshold are selected from adjacent frames as trajectory points of the feature points. All trajectory points are concatenated to generate a motion trajectory, and the simulation results of the motion trajectories of all feature points are output.

[0108] Step S33. Judge the simulation results of all trajectories and calculate the target likelihood ratio of the trajectory:

[0109]

[0110] Where L[P(k)] represents the target likelihood ratio of the trajectory, P(k) represents the pixel point of the trajectory in the kth frame, σ 2 is the variance of the Gaussian distribution obeyed by the background noise function f(t), U represents the number of pixels in the trajectory, and I(k) represents the brightness of the trajectory in the kth frame;

[0111] Step S34: Select the trajectory with the largest target likelihood ratio as the target trajectory of the feature point, predict the position of the target trajectory in the next frame of the image, and adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory.

[0112] Step S4. Using the AI ​​model to segment the static fractals and dynamic fractals in the visible light image stream, the target trajectory in the infrared image is color-converted and matched with the dynamic fractals in the visible light image using color as the standard. The matching errors in the horizontal and vertical directions of the trajectory are used as the horizontal and vertical deviations of the infrared image.

[0113] Step S4 includes:

[0114] Step S41: Segmenting the visible light image stream into static fractals and dynamic fractals, wherein the static fractals represent the portion of the visible light image stream where the color does not change, and the dynamic fractals represent the portion where the color changes;

[0115] Step S42. Use a grayscale fusion algorithm or a morphological image fusion algorithm to convert the target trajectory into a color trajectory, determine the position of the trajectory in the dynamic fractal through a color matching algorithm, and record the average matching error of the target trajectory in the horizontal and vertical directions as the horizontal and vertical deviations of the infrared image.

[0116] Step S5. Based on the horizontal and vertical deviations of the infrared image, bundle block adjustment is introduced to identify and calibrate the nonlinear geometric distortion in the infrared image through vertical feature correction technology.

[0117] Step S5 includes:

[0118] Step S51: For each frame in the infrared image stream, bundle block adjustment is introduced to calculate the calibration value of the pixel point (x, y) in the infrared image:

[0119]

[0120] Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical deviations, respectively; Dx and Dy represent the maximum values ​​of the pixel coordinates in the horizontal and vertical directions, respectively;

[0121] Step S52: Adjust the pixel positions in the infrared image according to the calculated calibration amount, eliminate the nonlinear geometric distortion in the infrared image, and output the corrected infrared image stream.

[0122] Example: The brightness of the feature point (1, 2) in the infrared image is 2.0, and the simulation results of the trajectory are [(1, 2), (2, 3), (3, 4)], [(1, 2), (0, 3), (2, 4)] and [(1, 2), (2, 1), (3, 0)], respectively. The background noise function obeys the normal distribution N(0, 0.04), and the brightness of points (2, 3) and (3, 4) are 1.8 and 2.1, respectively. The target likelihood ratio of the trajectory is 121.3, which is the maximum value among all trajectories. Therefore, trajectory 1 is taken as the target trajectory of the feature point.

[0123] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0124] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A high-precision image analysis method based on infrared thermal imaging, characterized in that: The method comprises the following steps: Step S1. When the infrared imager captures a preset number of frames, a visible light imaging device with the same resolution is used to simultaneously capture visible light images, and the infrared image and visible light image are stored separately to obtain an infrared image stream and a visible light image stream; Step S2: Temporal filtering is performed on the infrared image stream. The sum of squared errors of pixel brightness sequences between adjacent frames is used as the screening threshold. Pixels that do not meet the threshold are filtered out as noise points. The brightness difference of noise points in adjacent frames is used as the noise intensity to separate the background noise. Step S3. Calculate the diffraction expansion radius of each pixel in the infrared image. Pixels with radii above a threshold are considered feature points. Simulate the motion trajectory of the feature points by calculating their movement probability within adjacent frames. The trajectory with the maximum likelihood ratio is used as the target trajectory of the pixel. Adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory. Step S4. Using the AI ​​model to segment the static fractals and dynamic fractals in the visible light image stream, the target trajectory in the infrared image is color-converted and matched with the dynamic fractals in the visible light image using color as the standard. The matching errors in the horizontal and vertical directions of the trajectory are used as the horizontal and vertical deviations of the infrared image. Step S5. Based on the horizontal and vertical deviations of the infrared image, bundle block adjustment is introduced to identify and calibrate the nonlinear geometric distortion in the infrared image through vertical feature correction technology.

2. The high-precision image analysis method based on infrared thermal imaging according to claim 1, characterized in that: Step S1 includes: Step S11. Using a fusion device to connect the infrared imaging recognition and visible light imaging devices, the fusion device includes: a front-end detection device, an optical fiber transmission device, a timing chip, and a back-end control device, wherein the timing chip communicates with the image database in real time to count the infrared imaging frames; Step S12. Every time the infrared imaging device takes N image frames, the visible light imaging device takes one image frame, where N is a preset frame rate ratio. The infrared image and the visible light image are stored in the image database in chronological order to form an infrared image stream and a visible light image stream.

3. The high-precision image analysis method based on infrared thermal imaging according to claim 2, characterized in that: Step S2 includes: Step S21: Perform time domain filtering on each pixel position in the infrared image stream. If the pixel does not meet the threshold condition, the pixel is marked as a noise point. The threshold condition is: Where m represents the starting frame number, L represents the duration of the noise point, both m and L are constants, and m < N, L ≤ Nm-1, R(k) represents the brightness of the pixel in the kth frame, ft represents the average value of the standard Gaussian distribution function within the frame duration, A represents the maximum brightness of the pixel in all frames, C is the background constant of the infrared device, and T1 is the preset threshold; Step S22. Calculate the brightness difference of the noise points in adjacent frames to form a time domain sequence r = [r(1), r(2), ..., r(L-1)], where r(1) to r(L-1) represent the brightness difference between the 1st frame and the 2nd frame, and the brightness difference between the L-1th frame and the Lth frame, respectively. Use the Gaussian distribution function to fit the time domain sequence r to obtain the background noise function f(t) at the pixel point, where t represents time.

4. The high-precision image analysis method based on infrared thermal imaging according to claim 3, characterized in that: Step S3 includes: Step S31. Calculate the diffraction expansion radius of the pixel point (x, y). The calculation equation is: Where R is the brightness of the pixel in the visible light image, A1 is the brightness of the pixel in the infrared image, x and y represent the horizontal and vertical coordinates of the pixel in the infrared image, ε is the diffraction expansion radius of the pixel, i is the preset pixel offset, and e is the base of the natural logarithm; Step S32: Pixels with diffraction expansion radii above a threshold are selected as feature points. Using image flow analysis software, pixels with brightness differences from the feature points below a threshold are selected from adjacent frames as trajectory points of the feature points. All trajectory points are concatenated to generate a motion trajectory, and the simulation results of the motion trajectories of all feature points are output. Step S33. Judge the simulation results of all trajectories and calculate the target likelihood ratio of the trajectory: Where L[P(k)] represents the target likelihood ratio of the trajectory, P(k) represents the pixel point of the trajectory in the kth frame, σ 2 is the variance of the Gaussian distribution obeyed by the background noise function f(t), U represents the number of pixels in the trajectory, and I(k) represents the brightness of the trajectory in the kth frame; Step S34: Select the trajectory with the largest target likelihood ratio as the target trajectory of the feature point, predict the position of the target trajectory in the next frame of the image, and adjust the shooting angle of the infrared imaging device so that the lens follows the target trajectory.

5. The high-precision image analysis method based on infrared thermal imaging according to claim 4, characterized in that: Step S4 includes: Step S41: Segmenting the visible light image stream into static fractals and dynamic fractals, wherein the static fractals represent the portion of the visible light image stream where the color does not change, and the dynamic fractals represent the portion where the color changes; Step S42. Convert the target trajectory into a color trajectory using a grayscale fusion algorithm or a morphological image fusion algorithm. Determine the position of the trajectory in the dynamic fractal using a color matching algorithm. Record the average matching error of the target trajectory in the horizontal and vertical directions as the horizontal and vertical deviations of the infrared image. Step S5 includes: Step S51: For each frame in the infrared image stream, bundle block adjustment is introduced to calculate the calibration value of the pixel point (x, y) in the infrared image: Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical deviations, respectively; Dx and Dy represent the maximum values ​​of the pixel coordinates in the horizontal and vertical directions, respectively; Step S52: Adjust the pixel positions in the infrared image according to the calculated calibration amount, eliminate the nonlinear geometric distortion in the infrared image, and output the corrected infrared image stream.

6. A high-precision image analysis system based on infrared thermal imaging, characterized in that: The system includes the following modules: instrument imaging module, background filtering module, target recognition module, image fusion module and distortion correction module; The instrument imaging module is used to connect the infrared imaging recognition and visible light imaging devices. After the infrared imaging device continuously shoots a fixed number of frames, the visible light imaging device simultaneously shoots the target, the infrared imaging device obtains an infrared image stream, records the pixel grayscale of each frame in the infrared image stream, and the visible light imaging device obtains a visible light image and records the pixel color in the visible light image; The background filtering module is used to use the AI ​​image segmentation model to identify static classifications from adjacent frame images, perform time-domain filtering on the corresponding non-static classification parts in the infrared image stream, use the sum of squared errors of the pixel brightness sequences between adjacent frames as the screening threshold, filter out pixels that do not meet the threshold conditions as noise points, and use the difference between the average brightness of the noise point and the adjacent pixel points as the noise brightness to separate the background noise from the actual signal, and calculate the Gaussian distribution parameters of the background noise; The target recognition module is configured to calculate the diffraction spread radius of each pixel in the non-static fractal portion of the infrared image stream, select pixels with a diffraction spread radius greater than a threshold as feature points, simulate trajectories of the feature points in the infrared image stream, calculate the probability distribution and likelihood ratio of each trajectory based on Gaussian distribution parameters of noise, select the trajectory with the maximum likelihood ratio as the target trajectory of the pixel, and mark all feature points on the trajectory as moving points when the target trajectory meets a continuity condition; The image fusion module is used to take the moving point as the center and all pixels within the diffraction expansion radius constitute the target area, perform color transformation on the target area, determine the color value of the moving target point, compare it with the color value of each pixel in the corresponding target area of ​​the visible light image, select the pixel point with the closest color value as the corresponding point, and after the infrared image and the visible image are aligned, use the grayscale transformation algorithm to fuse the infrared image and the visible light image; The distortion correction module is used to determine the horizontal and vertical pixel offset values ​​of the infrared lens based on the corresponding offsets of the infrared image and the visible image, introduce the bundle method regional block adjustment through vertical feature point correction technology, establish a calibration model, correct the nonlinear geometric distortion in the infrared image, and output the distortion-corrected infrared image.

7. The high-precision image analysis system based on infrared thermal imaging according to claim 6, characterized in that: The instrument imaging module includes: an infrared imaging unit and a visible light imaging unit; The infrared imaging unit is used to control the infrared imaging device to continuously capture infrared images of the target and generate an infrared image stream; The visible light imaging unit is used to calculate the shooting frame number of the infrared device, and shoot the target simultaneously with the infrared imaging device every fixed frame number to generate a visible light image.

8. The high-precision image analysis system based on infrared thermal imaging according to claim 7, characterized in that: The background filtering module includes: an image segmentation unit, a time domain filtering unit and a noise separation unit; The image segmentation unit is used to segment the visible light image to segment the static fractals in adjacent visible light images; The time domain filtering unit is used to correspond the static classification to the infrared image and filter the pixel grayscale in the infrared image; The noise separation unit is used to identify noise points according to the time domain filtering result and separate the background noise.

9. The high-precision image analysis system based on infrared thermal imaging according to claim 8, characterized in that: The target recognition module includes: a feature extraction unit, a trajectory prediction unit and a dynamic recognition unit; The feature extraction unit is used to calculate the diffraction expansion radius of each pixel point and determine the feature point of the starting trajectory; The trajectory prediction unit is used to simulate the trajectory of the feature points to obtain the motion trajectory with the highest probability of the feature points; The dynamic recognition unit is used to determine the motion speed and motion direction of the feature point according to the length of the motion trajectory and the frame duration.

10. The high-precision image analysis system based on infrared thermal imaging according to claim 9, characterized in that: The image fusion module includes: a region contrast unit and a grayscale conversion unit; The regional comparison unit is used to compare the infrared image and the visible light image to determine the corresponding positions between the images; The grayscale conversion unit is used to convert the infrared grayscale image into a readable visible light image using a grayscale conversion algorithm; The distortion correction module includes: a deviation comparison unit and a geometric restoration unit; The deviation comparison unit is used to determine the horizontal and vertical deviations of the infrared images based on the corresponding positions between the images; The geometric restoration unit is used to establish a deviation correction model to eliminate nonlinear geometric distortion in infrared images.

Images