A thermal imaging image correction method and device

Abstract

The embodiment of the present invention provides a method and device for correcting a thermal imaging image, the method comprising: determining the motion state of a thermal imaging device corresponding to a thermal image to be corrected; determining noise interference pixels from the thermal image to be corrected using a target screening condition corresponding to the motion state; and correcting the thermal image to be corrected based on the noise interference pixels. Different motion states correspond to different target screening conditions, that is, corresponding noise interference pixels can be determined for different motion states of the thermal imaging device, and then the thermal image to be corrected can be corrected using the noise interference pixels corresponding to the motion state, thereby achieving an effect of accurately correcting thermal images to be corrected in different motion states, and improving the accuracy of image correction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a thermal imaging image correction method and device. Background Art

[0002] Infrared thermal imaging devices generally have non-uniform response problems, which are mainly manifested in the serious interference of fixed pattern noise in the image. Therefore, it is very necessary to correct the non-uniformity of the image. At present, the process of the method for non-uniform correction of images is as follows: for the images collected by the infrared thermal imaging device in a moving state, the edge detection operator is used to detect the edge of the image, and the detected edge is used as the target contour. In order to avoid the target contour being mistakenly introduced into the correction result, all edge positions are not calculated for fixed pattern noise, and the correction matrix that can offset the fixed pattern noise is obtained in the form of iteration for each frame of the image, and the correction matrix is ​​superimposed on the original image to achieve non-uniform correction.

[0003] For images collected by infrared thermal imaging equipment in a static state, the grayscale difference between each pixel of the image and its four neighboring pixels is calculated, and the fixed pattern noise is calculated for pixels whose grayscale difference is within a preset interval and their corresponding neighboring pixels. A correction matrix that can offset the fixed pattern noise is obtained in an iterative form for each frame of the image, and the correction matrix is ​​superimposed on the original image to achieve non-uniform correction.

[0004] However, for the image corresponding to the above-mentioned motion state, if there is a high-temperature target in the scene, the image edge in the low-temperature area will be difficult to detect, and there will be a problem of missed image edge detection. If the overall temperature of the scene is low, it will cause too many edges of the image to be detected, and there will be a problem of multiple image edge detection. The multiple image edge detection problem and the missed image edge detection problem will both cause the calculated correction matrix to have a large error, resulting in image correction errors. In addition, since the above-mentioned image correction method is to calculate the grayscale difference of each pixel of the entire image without distinction, this can easily cause the calculated correction matrix to include the target information in the image, which in turn causes ghosting in the corrected image. For the image corresponding to the above-mentioned static state, since the fixed graphic noise calculation is performed on the pixels whose grayscale difference is within the preset interval and their corresponding neighboring pixels, the accuracy of image correction is low. Therefore, the current image non-uniformity correction method has the problem of low image correction accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a thermal imaging image correction method and device to improve the accuracy of image correction.

[0006] In a first aspect, an embodiment of the present invention provides a thermal imaging image correction method, comprising:

[0007] Determining the motion state of the thermal imaging device corresponding to the thermal image to be corrected;

[0008] Determine noise interference pixels from the thermal image to be corrected using the target screening condition corresponding to the motion state; wherein the target screening conditions corresponding to different motion states are different;

[0009] The thermal image to be corrected is corrected according to the noise interference pixels.

[0010] Optionally, the method further includes:

[0011] Determining the neighborhood difference of each pixel in the thermal image to be corrected relative to its neighboring pixels;

[0012] The target screening conditions corresponding to different motion states have different requirements on the proximity difference between noise interference pixels and their neighboring pixels.

[0013] Optionally, the motion state includes at least two of the following motion states: a high-speed motion state, a low-speed motion state, and a stationary state;

[0014] The target screening condition includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the reference frame thermal image represent different targets, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0015] The target screening conditions corresponding to different motion states have different requirements for the proximity difference of noise interference pixels relative to their neighboring pixels, including: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state.

[0016] Optionally, the target screening condition corresponding to the high-speed motion state further includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the thermal image of the reference frame represent the same target, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the thermal image of the reference frame is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0017] In the target screening condition corresponding to the high-speed motion state, the preset difference threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset difference threshold corresponding to the same target.

[0018] Optionally, the target screening condition further includes: the sum of the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than a preset sum value threshold;

[0019] The target screening conditions corresponding to different motion states have different requirements for the proximity difference of noise interference pixels relative to their neighboring pixels, including: the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the stationary state.

[0020] Optionally, the target screening condition corresponding to the high-speed motion state further includes:

[0021] In the target screening condition corresponding to the high-speed motion state, the preset sum value threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset sum value threshold corresponding to the same target.

[0022] Optionally, the correcting the thermal image to be corrected according to the noise interference pixels includes:

[0023] For each pixel in the thermal image to be corrected whose neighborhood has a noise interference pixel, determining a neighborhood difference between the pixel and the noise interference pixel in its neighborhood;

[0024] If the neighboring difference satisfies a preset difference condition, the noise interference pixel is determined to be a pixel to be corrected; wherein the preset difference condition indicates that the pixel and the noise interference pixel in its neighborhood correspond to the same target, and different motion states correspond to different preset difference conditions;

[0025] Determining a correction parameter corresponding to the pixel to be corrected based on the neighboring difference;

[0026] The pixels to be corrected are corrected according to the correction parameters to obtain a corrected thermal image.

[0027] Optionally, the preset difference condition includes that the adjacent difference is not greater than a preset difference threshold, and the preset difference conditions corresponding to different motion states include: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state.

[0028] Optionally, determining the correction parameter corresponding to the pixel to be corrected based on the neighboring difference includes:

[0029] The correction parameter corresponding to the pixel to be corrected is determined according to the neighboring difference and the preset correction degree parameter corresponding to the motion state; wherein the preset correction degree parameters corresponding to different motion states are different.

[0030] Optionally, the different preset correction degree parameters corresponding to the different motion states include: the value of the preset correction degree parameter corresponding to the high-speed motion state is greater than the value of the preset correction degree parameter corresponding to the low-speed motion state, and / or the value of the preset correction degree parameter corresponding to the low-speed motion state is greater than the value of the preset correction degree parameter corresponding to the stationary state.

[0031] Optionally, determining the motion state of the thermal imaging device corresponding to the thermal image to be corrected includes:

[0032] Acquire multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected, wherein each group of angular velocities includes at least one of a swing angular velocity, a tilt angular velocity, and a flip angular velocity;

[0033] Calculate the range and sum of angular velocities for each type;

[0034] Based on the range and the sum, it is determined that the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state or a stationary state.

[0035] Optionally, determining, based on the range difference and the sum value, that the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state, or a stationary state includes:

[0036] If the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold, determining the motion state of the thermal imaging device as a high-speed motion state;

[0037] If the range is less than a first preset range threshold and the sum is less than a first preset sum threshold, and when the range is greater than a second preset range threshold and the sum is greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a low-speed motion state; wherein the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold;

[0038] If the range is not greater than a second preset range threshold and the sum is not greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a stationary state.

[0039] Optionally, the thermal image to be corrected is a target thermal image to be processed, or the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image.

[0040] Optionally, the method further includes:

[0041] Acquire an accumulated correction parameter obtained by accumulating correction parameters corresponding to each frame of thermal image before the target thermal image;

[0042] The target thermal image is preliminarily corrected based on the accumulated correction parameters to obtain the preliminarily corrected thermal image.

[0043] In a second aspect, an embodiment of the present invention provides a thermal imaging image correction device, comprising:

[0044] A motion state determination module, used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected;

[0045] An interference pixel determination module, used to determine noise interference pixels from the thermal image to be corrected by using the target screening condition corresponding to the motion state; wherein the target screening conditions corresponding to different motion states are different;

[0046] An image correction module is used to correct the thermal image to be corrected according to the noise interference pixels.

[0047] Optionally, the device further comprises:

[0048] A neighboring difference determination module is used to determine the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels; the target screening conditions corresponding to different motion states have different requirements for the neighboring difference of the noise interference pixel relative to its neighboring pixels;

[0049] The motion state includes at least two of the following motion states: a high-speed motion state, a low-speed motion state, and a stationary state;

[0050] The target screening condition includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the reference frame thermal image represent different targets, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0051] The target screening conditions corresponding to different motion states have different requirements for the neighboring difference of the noise interference pixel relative to its neighboring pixels, including: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state;

[0052] The target screening condition corresponding to the high-speed motion state also includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the thermal image of the reference frame represent the same target, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the thermal image of the reference frame is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0053] In the high-speed motion state corresponding target screening condition, the preset difference threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset difference threshold corresponding to the same target;

[0054] The target screening condition also includes: the sum of the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than a preset sum value threshold;

[0055] The target screening conditions corresponding to different motion states have different requirements for the neighboring difference of noise interference pixels relative to their neighboring pixels, including: the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the static state;

[0056] The target screening conditions corresponding to the high-speed motion state also include:

[0057] In the high-speed motion state corresponding target screening condition, the preset sum value threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset sum value threshold corresponding to the same target;

[0058] The image correction module comprises:

[0059] A difference determination submodule, for determining, for each pixel in the thermal image to be corrected, a noise interference pixel in its neighborhood, a neighborhood difference between the pixel and the noise interference pixel in its neighborhood;

[0060] A pixel determination submodule, configured to determine that the noise interference pixel is a pixel to be corrected if the neighboring difference satisfies a preset difference condition; wherein the preset difference condition indicates that the pixel and the noise interference pixel in its neighborhood correspond to the same target, and different motion states correspond to different preset difference conditions;

[0061] A correction parameter determination submodule, used to determine the correction parameter corresponding to the pixel to be corrected based on the neighboring difference;

[0062] An image correction submodule, used to correct the pixels to be corrected according to the correction parameters to obtain a corrected thermal image;

[0063] The preset difference condition includes that the proximity difference is not greater than a preset difference threshold, and the preset difference conditions corresponding to different motion states include: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state;

[0064] The correction parameter determination submodule is specifically used to determine the correction parameter corresponding to the pixel to be corrected according to the neighboring difference and the preset correction degree parameter corresponding to the motion state; wherein the preset correction degree parameters corresponding to different motion states are different;

[0065] The different preset correction degree parameters corresponding to the different motion states include: the value of the preset correction degree parameter corresponding to the high-speed motion state is greater than the value of the preset correction degree parameter corresponding to the low-speed motion state, and / or the value of the preset correction degree parameter corresponding to the low-speed motion state is greater than the value of the preset correction degree parameter corresponding to the static state;

[0066] The motion state determination module is specifically used to obtain multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected, wherein each group of angular velocities includes at least one of a swing angular velocity, a tilt angular velocity, and a flip angular velocity; calculate the range and sum between each type of angular velocity; and determine, based on the range and the sum, whether the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state, or a stationary state;

[0067] The motion state determination module is specifically used to determine the motion state of the thermal imaging device as a high-speed motion state if the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold; if the range is less than the first preset range threshold and the sum is less than the first preset sum threshold, and when the range is greater than a second preset range threshold and the sum is greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a low-speed motion state; wherein the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold; if the range is not greater than the second preset range threshold and the sum is not greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a stationary state;

[0068] The thermal image to be corrected is a target thermal image to be processed, or the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image;

[0069] The device also includes:

[0070] The preliminary correction module is used to obtain the accumulated correction parameters obtained by accumulating the correction parameters corresponding to each frame of thermal image before the target thermal image; based on the accumulated correction parameters, the target thermal image is preliminarily corrected to obtain the preliminary corrected thermal image.

[0071] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0072] Memory, used to store computer programs;

[0073] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0074] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0075] Beneficial effects of the embodiments of the present invention:

[0076] The method provided in the embodiment of the present invention is used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected; the noise interference pixels are determined from the thermal image to be corrected using the target screening conditions corresponding to the motion state; and the thermal image to be corrected is corrected based on the noise interference pixels. Different motion states correspond to different target screening conditions, that is, the corresponding noise interference pixels can be determined for different motion states of the thermal imaging device, and then the thermal image to be corrected can be corrected using the noise interference pixels corresponding to the motion state, thereby achieving the effect of accurately correcting the thermal image to be corrected in different motion states, and improving the accuracy of image correction. Of course, implementing any product or method of the present invention does not necessarily require all of the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0078] Figure 1 A flow chart of a thermal imaging image correction method provided by an embodiment of the present invention;

[0079] Figure 2 A flow chart of correcting a thermal image to be corrected provided by an embodiment of the present invention;

[0080] Figure 3 A schematic diagram of each pixel in the thermal image to be corrected;

[0081] Figure 4 A schematic diagram of thermal imaging image correction;

[0082] Fig. 5(a) is an outdoor thermal image to be corrected;

[0083] FIG5(b) is a corrected thermal image obtained by correcting the outdoor thermal image to be corrected shown in FIG5(a) using the thermal imaging image correction method provided by the present invention;

[0084] Figure 6(a) is another outdoor thermal image to be corrected;

[0085] FIG6(b) is a corrected thermal image obtained by correcting the outdoor thermal image to be corrected shown in FIG6(a) using the thermal imaging image correction method provided by the present invention;

[0086] FIG7( a ) is an indoor thermal image to be corrected;

[0087] FIG7( b ) is a corrected thermal image obtained by correcting the indoor thermal image to be corrected shown in FIG7( a ) using the thermal imaging image correction method provided by the present invention;

[0088] Figure 8(a) is another indoor thermal image to be corrected;

[0089] FIG8(b) is a corrected thermal image obtained by correcting the indoor thermal image to be corrected shown in FIG8(a) using the thermal imaging image correction method provided by the present invention;

[0090] Fig. 9 A structural diagram of a thermal imaging image correction device provided by an embodiment of the present invention;

[0091] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0093] When performing infrared imaging, infrared thermal imaging devices such as infrared thermal imaging cores and thermal imagers may have inconsistent responses of various pixels due to non-uniform response, uneven lens transmittance, or body radiation. This phenomenon will cause fixed pattern noise in the output infrared image. For thermal imaging cores without baffle structures, the fixed pattern noise of the output infrared image is also particularly serious. Among them, the fixed image noise of the image is generally manifested as pockmarks or vertical lines floating on the surface of the image. Therefore, it is very necessary to correct the fixed pattern noise of the image.

[0094] In order to improve the accuracy of image correction, embodiments of the present invention provide a thermal imaging image correction method, device, electronic device, computer-readable storage medium, and computer program product.

[0095] The following first introduces the thermal imaging image correction method provided by the embodiment of the present invention. The thermal imaging image correction method provided by the embodiment of the present invention can be applied to any electronic device with an image processing function, and is not specifically limited here.

[0096] Figure 1 A flow chart of a thermal imaging image correction method provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0097] S101, determining the motion state of a thermal imaging device corresponding to a thermal image to be corrected.

[0098] S102: Determine noise interference pixels from the thermal image to be corrected by using the target screening condition corresponding to the motion state.

[0099] Among them, different motion states correspond to different target screening conditions.

[0100] S103, correcting the thermal image to be corrected according to the noise interference pixels.

[0101] It can be seen that the method provided by the embodiment of the present invention is used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected; determine the noise interference pixels from the thermal image to be corrected using the target screening conditions corresponding to the motion state; and correct the thermal image to be corrected based on the noise interference pixels. Different motion states correspond to different target screening conditions, that is, the corresponding noise interference pixels can be determined for different motion states of the thermal imaging device, and then the noise interference pixels corresponding to the motion state can be used to correct the thermal image to be corrected, thereby achieving the effect of accurately correcting the thermal image to be corrected in different motion states, and improving the accuracy of image correction.

[0102] In the embodiment of the present invention, the thermal image to be corrected may be a target thermal image to be processed, or the thermal image to be corrected may be a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image.

[0103] The target thermal image may be an unprocessed original thermal image acquired by the thermal imaging device, or the target thermal image may be an image obtained by performing preliminary image processing on the original thermal image acquired by the thermal imaging device. The preliminary image processing may include image enhancement processing, image restoration processing, image segmentation processing, etc., which are not specifically limited here.

[0104] In a possible implementation, since the change in angular velocity of the thermal imaging device can reflect the movement of the thermal imaging device, the change in angular velocity of the thermal imaging device can be obtained, and the change in angular velocity can be used to determine the movement state of the thermal imaging device corresponding to the thermal image to be corrected. The movement state generally includes a high-speed movement state, a medium-speed movement state, a low-speed movement state, and a stationary state, etc.

[0105] Specifically, the above step of determining the motion state of the thermal imaging device corresponding to the thermal image to be corrected may include steps A1 to A3:

[0106] Step A1: Acquire multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected. The thermal imaging device corresponding to the thermal image to be corrected is a thermal imaging device that collects the original thermal image corresponding to the thermal image to be corrected. Each group of angular velocities may include at least one of a swing angular velocity, a tilt angular velocity, and a flip angular velocity.

[0107] In this embodiment, an angular velocity detection device, such as a gyroscope, can be used to detect multiple angular velocities of the thermal imaging device, such as three types of angular velocities: swing angular velocity, tilt angular velocity, and flip angular velocity. Among them, the swing angular velocity reflects the angular velocity of the thermal imaging device in the "left-right swing" direction, the tilt angular velocity reflects the angular velocity of the thermal imaging device in the "left-right tilt" direction, and the flip angular velocity reflects the angular velocity of the thermal imaging device in the "front-back flip" direction.

[0108] Specifically, in this embodiment, multiple groups of angular velocities detected by the gyroscope can be obtained from the thermal imaging device during the time period from the previous thermal image to the original thermal image corresponding to the thermal image to be corrected to the acquisition of the original thermal image. For example, the time when the thermal imaging device acquires the previous thermal image of the original thermal image is t1, and the time when the thermal imaging device acquires the original thermal image is t2. In the time period [t1, t2], the gyroscope detects three groups of angular velocities of the thermal imaging device. Then, this step can acquire multiple groups of angular velocities of the thermal imaging device in the time period [t1, t2]: (α1, β1, γ1), (α2, β2, γ2) and (α3, β3, γ3), where α1-α3 are swing angular velocities, β1-β3 are tilt angular velocities, and γ1-γ3 are flip angular velocities.

[0109] Step A2: Calculate the range and sum of the angular velocities for each type.

[0110] In this step, for each two groups of angular velocities, the difference between the angular velocities of the same type in the two groups can be calculated, and then the maximum difference of each type can be determined as the range between the angular velocities of that type. For example, for the swing angular velocity, the difference can be calculated: (α1-α2), (α1-α3) and (α2-α3), and the maximum difference can be determined as the range corresponding to the swing angular velocity; for the tilt angular velocity, the difference can be calculated: (β1-β2), (β1-β3) and (β2-β3), and the maximum difference can be determined as the range corresponding to the tilt angular velocity; for the flip angular velocity, the difference can be calculated: (γ1-γ2), (γ1-γ3) and (γ2-γ3), and the maximum difference can be determined as the range corresponding to the flip angular velocity.

[0111] For each type of angular velocity, multiple angular velocities of the type can be added together to obtain the sum of the multiple angular velocities of the type. For example, the sum of the swing angular velocity can be calculated as: (α1+α2+α3), the sum of the tilt angular velocity can be calculated as: (β1+β2+β3), and the sum of the flip angular velocity can be calculated as: (γ1+γ2+γ3).

[0112] Step A3: Based on the range and the sum, determine whether the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state or a stationary state.

[0113] The range may reflect the maximum change in angular velocity within a period from the previous thermal image of the original thermal image to the original thermal image, and the sum may reflect the distance traveled by the movement of the thermal imaging device within the period of collecting two frames of images.

[0114] Specifically, if the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold, the motion state of the thermal imaging device is determined to be a high-speed motion state;

[0115] If the range is less than a first preset range threshold and the sum is less than a first preset sum threshold, and if the range is greater than a second preset range threshold and the sum is greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a low-speed motion state;

[0116] If the range is not greater than a second preset range threshold and the sum is not greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a stationary state.

[0117] Among them, the first preset range threshold, the second preset range threshold, the first preset sum threshold and the second preset sum threshold can all be set according to actual application conditions, and are not specifically limited here, and the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold.

[0118] In an embodiment of the present invention, it is possible to first determine whether the motion state of the thermal imaging device is a high-speed motion state based on the first preset extreme difference threshold and the first preset sum value threshold. If not, it is then possible to continue to determine whether the motion state of the thermal imaging device is a low-speed motion state, and whether it is a stationary state, based on the second preset extreme difference threshold and the second preset sum value threshold.

[0119] It can be seen that in this embodiment, based on the multiple groups of angular velocity data output by the gyroscope, the motion state of the thermal imaging device can be divided into three types: high-speed motion state, low-speed motion state and static state. It can not only determine whether the thermal imaging device is in motion or non-motion state, but also determine the speed of the thermal imaging device, so as to achieve high-precision judgment of the motion state of the thermal imaging device. In this embodiment, for the situation that does not meet the high-speed motion state determination condition but the actual motion speed has reached high speed, that is, the high-speed motion state missed detection data, it is highly likely to meet the subsequent low-speed motion state determination condition. For the thermal image to be corrected in this case, it can also be corrected according to the low-speed motion state to achieve a certain correction effect. Similarly, for the situation that does not meet the low-speed motion state determination condition but is actually low speed, that is, the low-speed motion state missed detection data, the noise interference pixels screened out by the target screening condition corresponding to the static state can be used to correct the thermal image to be corrected, so as to achieve a certain correction effect and ensure the accuracy of the correction.

[0120] After determining the motion state of the thermal imaging device, in the above step S102, the target screening conditions corresponding to the motion state can be used to determine the noise interference pixels from the thermal image to be corrected. Among them, the target screening conditions corresponding to different motion states are different. Since the distribution of fixed image noise interference pixels in the corresponding thermal image to be corrected is different when the motion state of the thermal imaging device is different, therefore, for the thermal images to be corrected corresponding to different motion states, it is necessary to filter out the noise interference pixels in the thermal image to be corrected in combination with the motion state. Therefore, the target screening conditions corresponding to the motion state can be used to determine accurate noise interference pixels from the thermal image to be corrected. Then, the above step S103 can be executed, that is, the thermal image to be corrected is corrected according to the noise interference pixels.

[0121] In a possible implementation, since the difference in grayscale values ​​between noise interference pixels and their neighboring pixels has a certain regularity, for example, for fixed pattern noise interference pixels, whether the scene changes due to the movement of the thermal imaging device's movement, or the movement of the target in the scene when the movement is stationary, the position of the fixed pattern noise interference pixel is always fixed, and the difference in grayscale values ​​between the pixel where the fixed pattern noise interference pixel is located and the neighboring pixels changes little. Therefore, the noise interference pixel can be selected by the change in the grayscale value of each pixel in the previous and next frames of the thermal image to be corrected, and the change in the neighboring difference between it and the neighboring pixels. Therefore, in order to accurately screen out noise interference pixels, the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels can be determined, and the noise interference pixel can be determined from the thermal image to be corrected using the neighboring difference and the target screening condition.

[0122] In addition, when the thermal imaging device is in high-speed motion, the scene changes greatly, and the noise interference pixel selection is relatively easy and reliable. When the thermal imaging device is in low-speed motion, the scene changes slightly, and the noise interference pixel selection reliability is reduced. When the thermal imaging device is in a stationary state, the scene changes slightly, and the noise interference pixel selection is difficult and has low reliability. That is, the scene changes differently under different motion states. Based on this, the noise interference pixels in the corresponding thermal images to be corrected under different motion states are also different. Therefore, the requirements for the neighboring difference of the noise interference pixels relative to their neighboring pixels in the target screening conditions corresponding to different motion states are also different.

[0123] In this embodiment, the grayscale difference of each pixel in the thermal image to be corrected relative to its neighboring pixels can be determined as the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels. Alternatively, in this embodiment, the method described in the following steps B1-B2 can also be used to determine the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels:

[0124] Step B1: pre-set a neighboring difference matrix of all zeros; traverse the pixels of the thermal image to be corrected, calculate the difference in pixel values ​​between the currently traversed pixel and its neighboring pixels, and the sum of the difference in element values ​​between the element corresponding to the currently traversed pixel in the neighboring difference matrix and its neighboring elements, as the pixel difference between the currently traversed pixel and its neighboring pixels.

[0125] The neighboring pixels of a pixel may be the neighborhood pixels of the pixel, for example, the neighboring pixels of a pixel may be its 4-neighborhood pixels or 8-neighborhood pixels, etc. Alternatively, the neighboring pixels of a pixel may also be pixels whose distance from the pixel is within a preset pixel distance range, wherein the preset pixel distance may be set to the spacing between two adjacent pixels or twice the spacing between two adjacent pixels, etc., which is not limited here.

[0126] The neighborhood pixels of the pixel currently traversed may be: each of the four neighborhood pixels of the pixel currently traversed, that is, the pixels closest to the left, right, top, and bottom of the pixel currently traversed. The neighborhood elements may be: each of the four neighborhood elements of the element corresponding to the pixel currently traversed in the neighborhood difference matrix, that is, the elements closest to the left, right, top, and bottom of the element corresponding to the pixel currently traversed in the neighborhood difference matrix.

[0127] Specifically, in this implementation, the following formula can be used to calculate the difference in pixel values ​​between the currently traversed pixel and its neighboring pixels, and the sum of the element value differences between the element corresponding to the currently traversed pixel in the neighboring difference matrix and its neighboring elements, as the pixel difference between the currently traversed pixel and its neighboring pixels:

[0128] dif=Y(i,j)-Y neighbor +Correct_noise(i,j)-Correct_noise neighbor

[0129] Wherein, dif is the pixel difference, Y(i,j) is the pixel value of the currently traversed pixel (i,j) in the thermal image to be corrected, Correct_noise is the neighboring difference matrix, and Y neighbor is the pixel value of the neighboring pixel of the currently traversed pixel (i, j), Correct_noise(i, j) is the value of the element corresponding to the currently traversed pixel in the neighboring difference matrix, Correct_noise neighbor is the value of the neighboring element of the element corresponding to the currently traversed pixel in the neighboring difference matrix.

[0130] Step B2: Based on the pixel difference, the value of the element corresponding to the neighborhood pixel in the neighborhood difference matrix is ​​updated to obtain an updated neighborhood difference matrix, until all pixels of the thermal image to be corrected are traversed to obtain a target neighborhood difference matrix, then the neighborhood difference of each pixel in the thermal image to be corrected relative to its neighboring pixels is determined as the value of the corresponding element in the target neighborhood difference matrix.

[0131] In this implementation manner, the following formula may be specifically used to update the value of the element corresponding to the neighborhood pixel in the neighborhood difference matrix based on the pixel difference:

[0132] Correct_noise neighbor =Correct_noise neighbor +0.5*dif

[0133] When all pixels of the thermal image to be corrected are traversed, the updated target neighborhood difference matrix Correct_noise can be obtained, and the neighborhood difference of each pixel in the thermal image to be corrected relative to its neighboring pixels can be determined as the value of the corresponding element in the target neighborhood difference matrix.

[0134] In the embodiment of the present invention, the target screening condition corresponding to the motion state can be used to screen out noise interference pixels from the thermal image to be corrected.

[0135] In a possible implementation, whether the scene changes are caused by the movement of the thermal imaging device or the target moves in the scene when the movement is stationary, the position of the fixed pattern noise is always fixed, and the grayscale difference between the pixel where the fixed pattern noise is located and the surrounding pixels changes little. Therefore, on the basis that the grayscale of the pixel in the thermal image to be corrected changes with the grayscale of the pixel at the corresponding position of the reference frame thermal image, the pixel of the thermal image to be corrected should be transferred from one target to another target. The grayscale of the pixel in the thermal image to be corrected changes with the grayscale of the pixel at the corresponding position of the reference frame thermal image due to the different temperatures of the two targets, and the pixel of the thermal image to be corrected is determined as a noise interference pixel. Therefore, the target screening condition may include: the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than a preset difference threshold, and the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel. Wherein, the reference frame thermal image can usually be a thermal image of the previous frame of the thermal image to be corrected.

[0136] Since the grayscale difference between two pixels located in the same target in the thermal image to be corrected is always within the normal grayscale difference range, if the grayscale difference between the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position in the reference frame thermal image is greater than the preset grayscale difference threshold, it means that the grayscale difference between the pixel at the corresponding position in the thermal image to be corrected and the pixel at the corresponding position in the reference frame thermal image exceeds the normal grayscale difference range, that is, it can be determined that the pixel at the corresponding position in the thermal image to be corrected and the pixel at the corresponding position in the reference frame thermal image are located at different targets. The grayscale difference range is the preset grayscale difference threshold, therefore, when the grayscale difference between the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position in the reference frame thermal image is greater than the preset grayscale difference threshold corresponding to the motion state, it can be determined that the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position in the reference frame thermal image represent different targets.

[0137] In the embodiment of the present invention, since the position of the fixed pattern noise interference pixel of each frame of the thermal image to be corrected corresponding to the thermal imaging device is always fixed, and the difference in grayscale value between the pixel where the fixed pattern noise interference pixel is located and the adjacent pixel changes little, that is, the difference in the adjacent difference between the pixels at the corresponding positions between the two frames of images is also within a certain difference range. The difference range is the preset grayscale difference threshold. Therefore, when the difference between the adjacent difference of the pixel in the thermal image to be corrected and the adjacent difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold, and the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets, the pixel in the thermal image to be corrected can be determined as a noise interference pixel.

[0138] In the embodiment of the present invention, since the scene changes greatly when the thermal imaging device is in a high-speed motion state, the selection of noise interference pixels is relatively easy and reliable, and the pixels at the corresponding positions of the thermal image to be corrected and the thermal image of the reference frame can easily correspond to different targets, the preset grayscale difference threshold corresponding to the high-speed motion state can be set to be smaller, so that more noise interference pixels can be determined and the accuracy of image correction can be improved. When the thermal imaging device is in a low-speed motion state, the scene changes slightly, and the reliability of noise interference pixel selection is reduced. Therefore, the preset grayscale difference threshold corresponding to the low-speed motion state can be set to be larger, the threshold for selecting noise interference pixels is increased, the selection accuracy of the threshold for noise interference pixels is ensured, the problem of mistaken selection of noise interference pixels is avoided, and the accuracy of image correction is ensured. When the thermal imaging device is in a stationary state, the scene changes slightly, the selection of noise interference pixels is difficult and the reliability is low, therefore, the preset grayscale difference threshold corresponding to the stationary state can be set to be larger, the threshold for selecting noise interference pixels is further increased, the selection accuracy of the threshold for noise interference pixels is ensured, the problem of mistaken selection of noise interference pixels is avoided, and the accuracy of image correction is ensured.

[0139] Therefore, the preset grayscale difference thresholds corresponding to different motion states are different. In addition, since the scene changes greatly when the thermal imaging device is in a high-speed motion state, the noise interference pixel selection is relatively easy and reliable, and the scene changes slightly when the thermal imaging device is in a low-speed motion state, the reliability of noise interference pixel selection is reduced, and the scene changes slightly when the thermal imaging device is in a stationary state, the noise interference pixel selection is difficult and the reliability is low. The preset grayscale difference thresholds corresponding to different motion states may include: the preset grayscale difference threshold corresponding to the stationary state is greater than the preset grayscale difference threshold corresponding to the low-speed motion state, and / or the preset grayscale difference threshold corresponding to the low-speed motion state is greater than the preset grayscale difference threshold corresponding to the high-speed motion state. The specific values ​​of the preset grayscale difference thresholds corresponding to different motion states can be set according to the empirical values ​​of actual applications, and are not specifically limited here.

[0140] In the embodiment of the present invention, since the scene changes greatly when the thermal imaging device is in a high-speed motion state, the difference between the adjacent differences of the pixels at the corresponding positions between the thermal image to be corrected and the pixels at the corresponding positions of the reference frame thermal image is usually large, therefore, the preset difference threshold corresponding to the high-speed motion state can be set larger, so that more noise interference pixels can be determined, and the accuracy of image correction can be improved. When the thermal imaging device is in a low-speed motion state, the scene changes slightly, and the difference between the adjacent differences of the pixels at the corresponding positions between the thermal image to be corrected and the pixels at the corresponding positions of the reference frame thermal image is usually small, therefore, the preset difference threshold corresponding to the high-speed motion state can be set smaller, the threshold for selecting noise interference pixels can be increased, the problem of misselecting noise interference pixels can be avoided, and the accuracy of image correction can be ensured. When the thermal imaging device is in a stationary state, the scene changes slightly, and the difference between the adjacent differences of the pixels at the corresponding positions between the thermal image to be corrected and the pixels at the corresponding positions of the reference frame thermal image is usually smaller, therefore, the preset difference threshold corresponding to the stationary state can be set smaller, further increasing the threshold for selecting noise interference pixels, avoiding the problem of misselecting noise interference pixels, and ensuring the accuracy of image correction.

[0141] Therefore, the preset difference thresholds corresponding to different motion states are different, and the different requirements for the neighboring difference of noise interference pixels relative to their neighboring pixels in the target screening conditions corresponding to the different motion states may include: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or, the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the static state. The specific values ​​of the preset difference thresholds corresponding to different motion states can be set according to the empirical values ​​of actual applications, and are not specifically limited here.

[0142] In a possible implementation, since the scene changes greatly when the thermal imaging device is in a high-speed motion state, when the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position of the reference frame thermal image is within the preset grayscale difference threshold corresponding to the high-speed motion state, it is possible that the pixel is still moving on a larger and more uniform target, or it is also possible that the radiation amount of the target collected by the thermal image to be corrected and the reference frame thermal image is similar. Therefore, the target screening condition corresponding to the high-speed motion state may also include: the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image represent the same target, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel.

[0143] For example, the target in the thermal image to be corrected is a large gray background. When the pixel moves at high speed on the gray background, the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position in the reference frame thermal image is also within the preset grayscale difference threshold corresponding to the high-speed motion state. Then, when the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is less than the preset difference threshold, the pixel at the corresponding position in the thermal image to be corrected can be determined as a noise interference pixel.

[0144] In the embodiment of the present invention, when two pixels correspond to the same target, the grayscale values ​​of the two pixels are relatively close, and the neighboring differences between the two pixels and their own neighboring pixels are also relatively close. Therefore, when the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent the same target, the difference between the neighboring difference of the corresponding position in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is also relatively small. When the two pixels correspond to different targets, the grayscale values ​​of the two pixels are greatly different, and the neighboring differences between the two pixels and their own neighboring pixels are also greatly different. Therefore, when the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets, the difference between the neighboring difference of the corresponding position in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is large. Therefore, in the high-speed motion state corresponding target screening condition, the preset difference threshold corresponding to the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets is greater than the preset difference threshold corresponding to the same target.

[0145] In a possible implementation, since the neighboring differences of each pixel in the thermal image are all within a certain range, the sum of the neighboring differences of the pixels at corresponding positions of the two frames of thermal images is also within the noise intensity range. The noise intensity range is the preset sum value threshold, so the target screening condition may also include: the sum of the neighboring differences of the pixel in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position of the reference frame thermal image is less than the preset sum value threshold.

[0146] In the embodiment of the present invention, since the scene changes greatly when the thermal imaging device is in a high-speed motion state, the neighboring differences of the pixels at the corresponding positions between the thermal image to be corrected and the pixels at the corresponding positions of the reference frame thermal image are usually large, therefore, the sum of the neighboring differences of the pixels in the thermal image to be corrected and the neighboring differences of the pixels at the corresponding positions of the reference frame thermal image is also large, and the preset sum value threshold corresponding to the high-speed motion state can be set larger, so that more noise interference pixels can be determined, and the accuracy of image correction can be improved. When the thermal imaging device is in a low-speed motion state, the scene changes little, and the neighboring differences of the pixels at the corresponding positions between the thermal image to be corrected and the pixels at the corresponding positions of the reference frame thermal image are usually small, therefore, the sum of the neighboring differences of the pixels in the thermal image to be corrected and the neighboring differences of the pixels at the corresponding positions of the reference frame thermal image is also small, and the preset sum value threshold corresponding to the low-speed motion state can be set smaller, so as to increase the threshold for selecting noise interference pixels, avoid the problem of mistakenly selecting noise interference pixels, and ensure the accuracy of image correction. When the thermal imaging device is in a stationary state, the scene changes slightly, and the proximity differences of pixels at corresponding positions between the thermal image to be corrected and the pixels at corresponding positions of the reference frame thermal image are smaller. Therefore, the sum of the proximity differences of the pixel in the thermal image to be corrected and the proximity differences of the pixels at corresponding positions of the reference frame thermal image is also larger. The preset sum value threshold corresponding to the stationary state can be set smaller, further improving the threshold for selecting noise-interfered pixels, avoiding the problem of misselecting noise-interfered pixels, and ensuring the accuracy of image correction.

[0147] Therefore, the preset sum value thresholds corresponding to different motion states are different, and the target screening conditions corresponding to the different motion states have different requirements for the proximity difference of noise interference pixels relative to their neighboring pixels, including: the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or, the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the static state. The specific values ​​of the preset sum value thresholds corresponding to different motion states can be set according to the empirical values ​​of actual applications, and are not specifically limited here.

[0148] In a possible implementation, when two pixels correspond to the same target, the neighboring differences between the two pixels and their own neighboring pixels are relatively small. Therefore, when the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent the same target, the sum of the neighboring differences of the corresponding position in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the reference frame thermal image is also relatively small. When two pixels correspond to different targets, the neighboring differences between the two pixels and their own neighboring pixels are relatively large. Therefore, when the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets, the sum of the neighboring differences of the pixel at the corresponding position in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the reference frame thermal image is large. Therefore, the target screening condition corresponding to the high-speed motion state may also include: in the target screening condition corresponding to the high-speed motion state, the preset sum threshold corresponding to the grayscale of the pixel in the thermal image to be corrected and the grayscale of the pixel at the corresponding position of the reference frame thermal image represent different targets is greater than the preset sum threshold corresponding to the same target.

[0149] The following is an example of the target filtering conditions corresponding to different motion states combined with specific filtering condition formulas:

[0150] When the motion state is a high-speed motion state, if the pixel (i, j) in the thermal image to be corrected satisfies the following condition (1) or condition (2), the pixel is determined to be a noise interference pixel.

[0151] Condition (1):

[0152]

[0153] Wherein, abs() represents the absolute value operation, Y(i,j) is the pixel value (i.e., pixel grayscale) of the pixel (i,j) in the thermal image to be corrected, Y'(i,j) is the pixel value of the pixel at the corresponding position in the reference frame thermal image, Correct_noise(i,j) is the neighboring difference of the pixel, Correct_noise'(i,j) is the neighboring difference of the pixel at the corresponding position in the reference frame thermal image, and n 11 is the preset difference threshold corresponding to the high-speed motion state when the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position in the thermal image of the reference frame represent different targets, n 12 is the preset sum value threshold corresponding to the high-speed motion state when the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image represent different targets, n 13 It is the preset grayscale difference threshold corresponding to the high-speed motion state.

[0154] When the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position in the reference frame thermal image is greater than the preset grayscale difference threshold corresponding to the high-speed motion state, that is, abs(Y(i,j)-Y′(i,j))>n 13 When , it can be determined that the pixel in the thermal image to be corrected and the pixel at the corresponding position of the reference frame thermal image represent different targets. At the same time, when the absolute value of the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold value n 11 , and the absolute value of the sum of the neighboring differences of the pixel in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the thermal image of the reference frame is less than the preset sum threshold value n 12 , the pixel in the thermal image to be corrected can be determined as a noise interference pixel.

[0155] Condition (2):

[0156]

[0157] Among them, n 21 is the preset difference threshold corresponding to the high-speed motion state when the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position in the thermal image of the reference frame represent the same target, n 22 is the preset sum value threshold corresponding to the high-speed motion state when the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position in the thermal image of the reference frame represent the same target, n 23 is the preset grayscale difference threshold corresponding to the high-speed motion state, n 11 >n 21 .

[0158] In the high-speed motion state, when the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position in the reference frame thermal image is within the preset grayscale difference threshold range corresponding to the high-speed motion state, that is, abs(Y(i,j)-Y′(i,j))≤n 23 When , it can be determined that the pixel in the thermal image to be corrected and the pixel at the corresponding position of the reference frame thermal image represent the same target. At the same time, when the absolute value of the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold value n 21 , and the absolute value of the sum of the neighboring differences of the pixel in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the thermal image of the reference frame is less than the preset sum threshold value n 22 , the pixel in the thermal image to be corrected can be determined as a noise interference pixel.

[0159] When the motion state is a low-speed motion state, if the pixel (i, j) in the thermal image to be corrected satisfies the following condition (3), it can be determined that the pixel is a noise interference pixel.

[0160] Condition (3):

[0161]

[0162] Among them, n 31 is the preset difference threshold corresponding to the low-speed motion state, n 32 is the preset sum threshold corresponding to the low-speed motion state, n 33 is the preset grayscale difference threshold corresponding to the low-speed motion state, n 31 <n 21 , n 32 <n 22 , n 33 >n 23 , n 13 =n 23 .

[0163] In the low-speed motion state, pixels with small grayscale differences are not allowed to be noise interference pixels. When the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position in the reference frame thermal image is greater than the preset grayscale difference threshold corresponding to the low-speed motion state, that is, abs(Y(i,j)-Y′(i,j))>n 33 When , it can be determined that the pixel in the thermal image to be corrected and the pixel at the corresponding position of the reference frame thermal image represent different targets. At the same time, when the absolute value of the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold value n 31 , and the absolute value of the sum of the neighboring differences of the pixel in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the thermal image of the reference frame is less than the preset sum threshold value n 32 , the pixel in the thermal image to be corrected can be determined as a noise interference pixel.

[0164] When the motion state is a stationary state, if the pixel (i, j) in the thermal image to be corrected satisfies the following condition (4), it can be determined that the pixel is a noise interference pixel.

[0165] Condition (4):

[0166]

[0167] Among them, n 41 is the preset difference threshold corresponding to the static state, n 42 is the preset sum threshold corresponding to the static state, n 43 is the preset grayscale difference threshold corresponding to the static state, n41 <n 31 , n 42 <n 32 , n 43 >n 33 .

[0168] When the motion state is a static state, if the pixel satisfies the above condition (4), it means that only weak fixed pattern noise can be corrected in the static state, and the grayscale difference change here is caused by the target motion. When the grayscale difference between the pixel grayscale of the pixel in the thermal image to be corrected and the pixel grayscale of the corresponding position in the reference frame thermal image is greater than the preset grayscale difference threshold corresponding to the static state, that is, abs(Y(i,j)-Y′(i,j))>n 43 When , it can be determined that the pixel in the thermal image to be corrected and the pixel at the corresponding position of the reference frame thermal image represent different targets. At the same time, when the absolute value of the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than the preset difference threshold value n 41 , and the absolute value of the sum of the neighboring differences of the pixel in the thermal image to be corrected and the neighboring differences of the pixel at the corresponding position in the thermal image of the reference frame is less than the preset sum threshold value n 42 , the pixel in the thermal image to be corrected can be determined as a noise interference pixel.

[0169] It can be seen that in this embodiment, the three motion states of the thermal imaging device correspond to different target screening conditions. For different motion states, the corresponding noise interference pixel selection mode can be entered, and the three modes are basically the same, only the corresponding target screening conditions are different. The screening framework is simple and easy to implement, and the amount of calculation is small. Combining the target screening condition with the motion state of the thermal imaging device, when screening noise interference pixels, the characteristics of fixed pattern noise being fixed in position and additive noise are utilized, which can realize the rapid screening of noise interference pixels, reduce the amount of calculation in the screening process, and reduce the screening error rate.

[0170] After determining the noise interference pixels in the thermal image to be corrected, in the above step S103, in order to exclude the noise interference pixels that are mistakenly selected, the neighboring differences between each pixel in the thermal image to be corrected with the noise interference pixels in its neighborhood and the noise interference pixels in its neighborhood can be used to further screen out the pixels to be corrected from the noise interference pixels, and then, the corresponding correction parameters can be determined for the pixels to be corrected, and then the correction parameters are used to correct the pixels to be corrected, and the corrected thermal image is obtained. The correction parameters are grayscale parameters that can correct the grayscale values ​​of the pixels to be corrected.

[0171] In a possible implementation, since the neighborhood difference between each pixel in the same target in the thermal image to be corrected is always within the preset difference threshold, when the neighborhood difference between the pixel and the noise interference pixel in its neighborhood is greater than the preset difference threshold, it indicates that the pixel and the noise interference pixel in its neighborhood are not located on the same target. Therefore, the excessive difference in the neighborhood difference between the noise interference pixel and the pixel in its neighborhood is not caused by fixed image noise, but because the pixel and the noise interference pixel in its neighborhood are located on different targets. The noise interference pixel is actually misjudged as a noise interference pixel, and there is no need to correct the noise interference pixel. When the neighborhood difference between the pixel and the noise interference pixel in its neighborhood is not greater than the preset difference threshold, it indicates that the pixel and the noise interference pixel in its neighborhood correspond to the same target. Therefore, the excessive difference in the neighborhood difference between the noise interference pixel and the pixel in its neighborhood is caused by fixed image noise, so the noise interference pixel is determined as a pixel to be corrected, and the pixel to be corrected needs to be corrected later. Therefore, the neighboring differences between each pixel with noise interference pixels in the neighborhood of the thermal image to be corrected and the noise interference pixels in its neighborhood can be used to further screen out the pixels to be corrected from the noise interference pixels, and then the corresponding correction parameters can be determined for the pixels to be corrected, and then the correction parameters can be used to correct the pixels to be corrected, and the corrected thermal image can be obtained. Specifically, Figure 2 A flow chart of correcting a thermal image to be corrected provided by an embodiment of the present invention, such as Figure 2 As shown, the correcting the thermal image to be corrected according to the noise interference pixels may include:

[0172] S201 , for each pixel in the thermal image to be corrected whose neighborhood has a noise interference pixel, determine a neighborhood difference between the pixel and the noise interference pixel in its neighborhood.

[0173] Specifically, a matrix of all-zero correction parameters can be predefined. For each pixel in the thermal image to be corrected that has a noise interference pixel in its neighborhood, the pixel difference between the pixel and the noise interference pixel in its neighborhood and the sum of the element differences between the corresponding position element in the correction parameter matrix and the element in its neighborhood can be calculated, and the sum is used as the neighborhood difference between the pixel and the noise interference pixel in its neighborhood.

[0174] S202: If the neighboring difference satisfies a preset difference condition, determining the noise interference pixel as a pixel to be corrected.

[0175] The preset difference condition indicates that the pixel and the noise interference pixel in its neighborhood correspond to the same target, and different motion states correspond to different preset difference conditions.

[0176] In the embodiment of the present invention, since the neighboring differences between pixels in the same target in the thermal image to be corrected are always within a certain difference range, if the neighboring difference between the pixel and the noise interference pixel in its neighborhood is not within the difference range, it means that the pixel and the noise interference pixel in its neighborhood are not located on the same target, that is, the difference between the neighboring differences between the noise interference pixel and the pixel in its neighborhood is too large not due to fixed image noise, but because the pixel and the noise interference pixel in its neighborhood are located on different targets, and the noise interference pixel is actually misjudged as a noise interference pixel. The difference range is the preset difference threshold, so the preset difference condition can include that the neighboring difference is not greater than the preset difference threshold.

[0177] Furthermore, since the scene changes greatly when the thermal imaging device is in high-speed motion, the neighboring differences of each pixel in the same target in the thermal image to be corrected are also large. Therefore, the preset difference threshold corresponding to the high-speed motion state can be set larger, so that more interference pixels to be corrected can be determined, that is, more pixels to be corrected can be corrected, thereby improving the accuracy of image correction. When the thermal imaging device is in low-speed motion, the scene changes slightly, and the neighboring differences of each pixel in the same target in the thermal image to be corrected are usually small. Therefore, the preset difference threshold corresponding to the low-speed motion state can be set smaller, thereby increasing the threshold for screening pixels to be corrected and ensuring the accuracy of image correction. When the thermal imaging device is in a stationary state, the scene changes slightly, and the neighboring differences of each pixel in the same target in the thermal image to be corrected are usually smaller. Therefore, the preset difference threshold corresponding to the stationary state can be set smaller, thereby further increasing the threshold for screening pixels to be corrected and ensuring the accuracy of image correction. Therefore, the preset difference condition may include that the proximity difference is not greater than the preset difference threshold, and the preset difference conditions corresponding to different motion states include: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the static state. The specific values ​​of the preset difference thresholds corresponding to different motion states can be set according to the empirical values ​​of actual applications, and are not specifically limited here.

[0178] Since the neighborhood differences between the pixels in the same target in the thermal image to be corrected are always within the preset difference threshold, when the neighborhood difference between the pixel and the noise interference pixel in its neighborhood is greater than the preset difference threshold, it means that the pixel and the noise interference pixel in its neighborhood are not located on the same target. Therefore, the excessive difference in the neighborhood difference between the noise interference pixel and the pixel in its neighborhood is not caused by fixed image noise, but because the pixel and the noise interference pixel in its neighborhood are located on different targets. The noise interference pixel is actually misjudged as a noise interference pixel, and there is no need to correct the noise interference pixel. When the neighborhood difference between the pixel and the noise interference pixel in its neighborhood is not greater than the preset difference threshold, it means that the pixel and the noise interference pixel in its neighborhood correspond to the same target. Therefore, the excessive difference in the neighborhood difference between the noise interference pixel and the pixel in its neighborhood is caused by fixed image noise, so the noise interference pixel is determined as a pixel to be corrected, and the pixel to be corrected needs to be corrected later.

[0179] S203: Determine a correction parameter corresponding to the pixel to be corrected based on the neighboring difference.

[0180] The correction parameter is a grayscale parameter that can correct the grayscale value of the pixel to be corrected.

[0181] Since the proximity difference between each pixel in the same target in the thermal image to be corrected and its own neighborhood is within a certain range, the proximity difference between a normal pixel and its own neighborhood pixel is usually within the proximity difference range, while the proximity difference between the pixel to be corrected and its neighborhood pixel is greater than the proximity difference between the normal pixel and its own neighborhood pixel, and is outside the proximity difference range. Therefore, the proximity difference between the pixel to be corrected and its neighborhood pixel can be used to determine the corresponding correction parameter for correcting the gray value of the pixel to be corrected, so that the proximity difference between the pixel to be corrected and its neighborhood pixel is within the normal proximity difference range.

[0182] The neighborhood difference between the pixel and the noise interference pixel in its neighborhood reflects the grayscale difference between the noise interference pixel and the normal pixel. Therefore, for each pixel to be corrected, the product of the neighborhood difference and the preset coefficient and the sum of the element values ​​at the corresponding position in the matrix of the correction parameter can be determined as the correction parameter corresponding to the pixel to be corrected. The preset coefficient can be specifically set according to the actual application scenario, for example, it can be set to 0.5 or 0.6.

[0183] If there are multiple neighboring pixels of the pixel to be corrected whose neighboring differences with the pixel to be corrected meet the preset difference condition, the correction parameters corresponding to the pixel to be corrected can be updated in sequence according to the order of calculating the neighboring differences between the pixel and the noise interference pixels in its neighborhood, until the neighboring differences between each neighboring pixel that meets the preset difference condition and the pixel to be corrected participate in the calculation or update of the correction parameters corresponding to the pixel to be corrected.

[0184] For example, Figure 3 A schematic diagram of each pixel in the thermal image to be corrected, such as Figure 3 As shown, pixel e is a noise interference pixel, and pixels a, b, c, and d are neighborhood pixels of the noise interference pixel e.

[0185] When the neighborhood difference between pixel a and noise interference pixel e, the neighborhood difference between pixel b and noise interference pixel e, the neighborhood difference between pixel c and noise interference pixel e, or the neighborhood difference between pixel d and noise interference pixel e meets the preset difference condition, the noise interference pixel e can be selected as the pixel to be corrected.

[0186] if Figure 3 In the method, the neighborhood difference between pixel a and the noise interference pixel e, the neighborhood difference between pixel b and the noise interference pixel e, the neighborhood difference between pixel c and the noise interference pixel e, and the neighborhood difference between pixel d and the noise interference pixel e all meet the preset difference condition, that is, the noise interference pixel e is the pixel to be corrected. At the same time, the neighborhood difference between each pixel and the noise interference pixel e in its neighborhood is calculated in sequence of pixel a, pixel b, pixel c, and pixel d. Then, after the neighborhood difference between pixel a and the pixel to be corrected is calculated, the correction parameter corresponding to the pixel to be corrected can be determined based on the neighborhood difference between pixel a and the pixel to be corrected. Then, after the neighborhood difference between pixel b and the pixel to be corrected is calculated, the correction parameter corresponding to the pixel to be corrected is updated based on the neighborhood difference between pixel b and the pixel to be corrected. Then, after the neighborhood difference between pixel c and the pixel to be corrected is calculated, the correction parameter corresponding to the pixel to be corrected is updated based on the neighborhood difference between pixel c and the pixel to be corrected. Then, after the neighborhood difference between pixel d and the pixel to be corrected is calculated, the correction parameter corresponding to the pixel to be corrected is updated based on the neighborhood difference between pixel d and the pixel to be corrected.

[0187] S204, correcting the pixels to be corrected according to the correction parameters to obtain a corrected thermal image.

[0188] Specifically, the correction parameter and the pixel value of the pixel to be corrected can be used as the corrected pixel value of the corresponding pixel in the thermal image to be corrected. When the pixel value of each pixel to be corrected in the thermal image to be corrected is replaced with the corrected pixel value, a corrected thermal image is obtained.

[0189] In a possible implementation, determining the correction parameter corresponding to the pixel to be corrected based on the neighboring difference includes: determining the correction parameter corresponding to the pixel to be corrected according to the neighboring difference and a preset correction degree parameter corresponding to the motion state.

[0190] Since the scene changes greatly when the thermal imaging device is in high-speed motion, the noise interference pixel selection is less difficult and reliable. Therefore, the preset correction degree parameter can be set to a larger value, which can speed up the correction speed and improve the correction efficiency without affecting the correction accuracy. When the thermal imaging device is in low-speed motion, the scene changes little, and the reliability of noise interference pixel selection is reduced. Therefore, the preset correction degree parameter can be set to a smaller value, giving priority to ensuring the correction accuracy. When the thermal imaging device is in a stationary state, the scene changes slightly, and the noise interference pixel selection is difficult and unreliable. Therefore, the preset correction degree parameter can be set to a smaller value, giving priority to ensuring the correction accuracy. Therefore, the preset correction degree parameters corresponding to different motion states are different, and the value of the preset correction degree parameter is larger when the thermal imaging device is in a high-speed motion state, the preset correction degree parameter is moderate when the thermal imaging device is in a low-speed motion state, and the preset correction degree parameter is extremely small when the thermal imaging device is in a stationary state.

[0191] Specifically, for each pixel to be corrected, the product of the neighboring difference, the preset coefficient and the preset correction degree parameter may be determined as the correction parameter corresponding to the pixel to be corrected.

[0192] The following steps C1-C6 combine specific formulas to illustrate the calculation method of the correction parameters:

[0193] Step C1, defining a matrix of all-zero correction parameters.

[0194] Step C2, traverse each pixel in the thermal image to be corrected whose neighborhood has noise interference pixels, and use the following formula to calculate the neighborhood difference between the currently traversed pixel and the noise interference pixels in its neighborhood:

[0195] dif=Y(i,j)-Y neighbor +Correct1(i,j)-Correct1 neighbor

[0196] Where, dif is the neighborhood difference between the currently traversed pixel and the noise interference pixel in its neighborhood, Y(i,j) is the pixel value of the currently traversed pixel (i,j) of the thermal image to be corrected, and Y neighbor is the pixel value of the noise interference pixel in the neighborhood of the currently traversed pixel (i, j), Correct1 is the matrix of correction parameters, Correct1(i, j) is the value of the element corresponding to the currently traversed pixel in the matrix of correction parameters, Correct1 neighboris the value of the element in the correction parameter matrix corresponding to the noise interference pixel in the neighborhood of the currently traversed pixel (i, j).

[0197] Step C3, determining whether the proximity difference is greater than a preset difference threshold corresponding to the motion state, if yes, executing step C4, otherwise executing step C5.

[0198] Step C4, setting the preset difference threshold to 0.

[0199] Step C5, using the following formula, according to the preset difference threshold and the preset correction degree parameter corresponding to the motion state, to update the correction parameter matrix:

[0200] Correct1 neighbor =Correct1 neighbor +k*dif*lr

[0201] Wherein, dif is the neighborhood difference between the currently traversed pixel and the noise interference pixel in its neighborhood, lr is the preset correction degree parameter corresponding to the motion state, and Correct1 on the right side of the equation is neighbor is the value of the element at the position corresponding to the noise interference pixel in the correction parameter matrix. Correct1 on the right side of the equation neighbor is the matrix of the updated correction parameters. According to the formula, a non-zero dif value can be calculated only when there are noise interference pixels in the neighborhood pixels of the currently traversed pixel, and the values ​​of the neighborhood elements of the Correct1 matrix can be updated. Otherwise, the values ​​of the neighborhood elements of the Correct1 matrix cannot be updated. Therefore, in this embodiment, the pixels in the neighborhood of the thermal image to be corrected where there are noise interference pixels can be found according to the noise interference pixels and traversed, thereby reducing the amount of calculation for calculating the target correction matrix.

[0202] Step C6: When all pixels in the neighborhood of the thermal image to be corrected that have noise interference are traversed, a target matrix of the final updated correction parameters can be obtained. The correction parameters of the pixels to be corrected are the element values ​​at the corresponding positions in the target matrix.

[0203] In the embodiment of the present invention, different motion states correspond to different preset correction degree parameters. Specifically, the different preset correction degree parameters corresponding to different motion states may include: the value of the preset correction degree parameter corresponding to the high-speed motion state is greater than the value of the preset correction degree parameter corresponding to the low-speed motion state, and / or the value of the preset correction degree parameter corresponding to the low-speed motion state is greater than the value of the preset correction degree parameter corresponding to the stationary state.

[0204] In the embodiment of the present invention, since the correction effects corresponding to each frame of the thermal image to be corrected can be accumulated, that is, the correction effect of the current thermal image to be corrected will affect the correction of the next frame of the thermal image to be corrected, therefore, as the number of corrected images increases, it is necessary to reduce the value of the preset correction degree parameter to reduce the correction effect of the thermal image to be corrected on the correction effect of the next frame of the thermal image to be corrected, so as to avoid the correction effect of the thermal image to be corrected on the correction effect of the next frame of the thermal image to be corrected. The correction accuracy of the next frame of the thermal image to be corrected is affected. Therefore, when the thermal imaging device is in a high-speed motion state, the preset correction degree parameter can decay as the number of correction images increases. When the thermal imaging device is in a low-speed motion state, the preset correction degree parameter can also decay as the number of correction images increases. When the thermal imaging device is in a stationary state, the preset correction degree parameter usually does not decay.

[0205] Specifically, when the thermal imaging device is in a high-speed motion state, the preset correction degree parameter can be attenuated as the number of corrected images increases, and the attenuation rule can be as follows: in the high-speed motion state, for every 128 frames of images corrected cumulatively, the corresponding preset correction degree parameter can be attenuated to 1 / 2 of the original, and under normal circumstances, it can be attenuated up to 3 times.

[0206] When the thermal imaging device is in a low-speed motion state, the preset correction degree parameter can decay as the number of corrected images increases, and the attenuation rule can be as follows: in the low-speed motion state, for every cumulative correction of 128 frames of images, the corresponding preset correction degree parameter can decay to 1 / 2 of the original, and under normal circumstances it can decay up to 3 times.

[0207] It can be seen that in the present embodiment, the calculation of the correction parameters utilizes the motion state of the thermal imaging device, the upper limit of the dif value in the high-speed motion state and the high-speed motion state is large, and the upper limit of the dif in the static state is small, and each dif value is set to 0 when it exceeds the upper limit, and the preset correction degree parameter corresponding to the high-speed motion state is greater than the preset correction degree parameter corresponding to the low-speed motion state, and the preset correction degree parameter corresponding to the low-speed motion state is greater than the preset correction degree parameter corresponding to the static state, and the high-speed and low-speed motion states contain a preset correction degree parameter attenuation mechanism, and the correction parameter calculation also utilizes noise interference pixels, that is, only pixels with noise interference pixels in the neighborhood can be updated, and the central pixel does not need to be a noise interference pixel, which further reduces the amount of calculation of the correction parameters and improves the image correction efficiency.

[0208] In a possible implementation, the step of performing preliminary correction on the target thermal image to obtain a preliminary corrected thermal image may include the following steps D1-D2:

[0209] Step D1: Obtaining an accumulated correction parameter obtained by accumulating correction parameters corresponding to each frame of thermal image before the target thermal image.

[0210] Step D2: Preliminary correction is performed on the target thermal image based on the accumulated correction parameters to obtain the preliminary corrected thermal image.

[0211] Specifically, Figure 4 A schematic diagram of thermal imaging image correction, see Figure 4 After reading in the target thermal image Y, the target thermal image Y can be preliminarily corrected using the cumulative correction parameter Correct to offset the fixed pattern noise in the target thermal image Y and obtain a preliminary corrected thermal image Y1; if the target thermal image Y is the first frame image, the all-zero matrix of the cumulative correction parameter can be superimposed on the target thermal image Y to obtain a preliminary corrected thermal image Y1.

[0212] If the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image, such as Figure 4 As shown, for the preliminary corrected thermal image Y1, noise interference pixels can be selected to calculate the correction parameter Correct1, and then the preliminary corrected thermal image Y1 can be corrected with the correction parameter Correct1 to obtain the corrected thermal image Y2 output. At the same time, after obtaining the correction parameter Correct1, the correction parameter Correct1 can be accumulated to the accumulated correction parameter Correct to update the accumulated correction parameter Correct, and then the updated accumulated correction parameter Correct can be used to perform preliminary correction on the next frame of thermal image to be corrected.

[0213] Among them, the accumulated correction parameter Correct is obtained by accumulating the correction parameters corresponding to each frame of thermal image before the target thermal image Y. For example, the correction parameter Correct1 corresponding to the target thermal image Y can be accumulated into Correct (Correct = Correct + Correct1) to perform preliminary correction on the next frame of thermal image to be corrected, so as to offset the fixed pattern noise in the next frame of thermal image to be corrected.

[0214] Moreover, as the number of corrected thermal images increases, each motion state updates the cumulative correction parameter Correct with a lower correction degree parameter to monitor the newly added fixed pattern noise and continuously optimize the image. At this time, the adjacent differences between the pixels of each player in the current corrected thermal image Y2 and the preliminary corrected thermal image Y1 need to be retained for use in determining the noise interference pixels of the next frame of the thermal image to be corrected.

[0215] In one embodiment, since it is necessary to use the proximity differences of pixels at corresponding positions of the thermal image to be corrected and the reference frame thermal image when determining the noise interference pixels of the thermal image to be corrected, after obtaining the proximity differences between the pixels of each team member of the current corrected thermal image Y2 and the preliminary corrected thermal image Y1, the difference between the proximity differences and the correction parameters can also be determined as the proximity differences between the corresponding pixels in the corrected thermal image and its neighboring pixels, which is used to determine the noise interference pixels in the next frame of thermal image to be corrected.

[0216] It can be seen that in the above implementation, the correction parameters calculated for each frame of the thermal image to be corrected are accumulated and used to perform preliminary correction on the next frame of the thermal image to be corrected, which not only improves the fault tolerance rate, but also corrects the previous correction error, ensuring the correctness of the general direction of the correction. In addition, the motion state is combined with the target screening condition to jointly determine the noise interference pixels of the preliminary correction thermal image, and then the noise interference pixels of the preliminary correction thermal image are used to perform fine correction on the preliminary correction thermal image, which greatly improves the accuracy of image correction.

[0217] FIG5(a) is an outdoor thermal image to be corrected, and FIG5(b) is a corrected thermal image obtained by correcting the outdoor thermal image to be corrected shown in FIG5(a) using the thermal imaging image correction method provided by the present invention. By comparing FIG5(a) and FIG5(b), it can be seen that the fixed pattern noise such as pockmarks and vertical lines in the corrected thermal image shown in FIG5(b) is corrected and removed compared with the thermal image to be corrected shown in FIG5(a).

[0218] FIG6(a) is another outdoor thermal image to be corrected, and FIG6(b) is a corrected thermal image obtained by correcting the outdoor thermal image to be corrected shown in FIG6(a) using the thermal imaging image correction method provided by the present invention. By comparing FIG6(a) and FIG6(b), it can be seen that the fixed pattern noise such as pockmarks and vertical lines in the corrected thermal image shown in FIG6(b) is corrected and removed compared with the thermal image to be corrected shown in FIG6(a).

[0219] FIG7(a) is an indoor thermal image to be corrected, and FIG7(b) is a corrected thermal image obtained by correcting the indoor thermal image to be corrected shown in FIG7(a) using the thermal imaging image correction method provided by the present invention. By comparing FIG7(a) and FIG7(b), it can be seen that the fixed pattern noise such as pockmarks and vertical lines in the corrected thermal image shown in FIG7(b) is corrected and removed compared with the thermal image to be corrected shown in FIG7(a).

[0220] FIG8(a) is another indoor thermal image to be corrected, and FIG8(b) is a corrected thermal image obtained by correcting the indoor thermal image to be corrected shown in FIG8(a) using the thermal imaging image correction method provided by the present invention. By comparing FIG8(a) and FIG8(b), it can be seen that the fixed pattern noise such as pockmarks and vertical lines in the corrected thermal image shown in FIG8(b) is corrected and removed compared with the thermal image to be corrected shown in FIG8(a).

[0221] Corresponding to the above-mentioned thermal imaging image correction method, an embodiment of the present invention further provides a thermal imaging image correction device. The thermal imaging image correction device provided by the embodiment of the present invention is introduced below. Fig. 9 As shown, a thermal imaging image correction device, the device comprising:

[0222] A motion state determination module 901 is used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected;

[0223] The interference pixel determination module 902 is used to determine noise interference pixels from the thermal image to be corrected by using the target screening condition corresponding to the motion state; wherein the target screening conditions corresponding to different motion states are different;

[0224] The image correction module 903 is used to correct the thermal image to be corrected according to the noise interference pixels.

[0225] The device provided by the embodiment of the present invention is used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected; the noise interference pixels are determined from the thermal image to be corrected using the target screening conditions corresponding to the motion state; and the thermal image to be corrected is corrected according to the noise interference pixels. Different motion states correspond to different target screening conditions, that is, the corresponding noise interference pixels can be determined for different motion states of the thermal imaging device, and then the thermal image to be corrected is corrected using the noise interference pixels corresponding to the motion state, thereby achieving the effect of accurately correcting the thermal image to be corrected in different motion states, and improving the accuracy of image correction.

[0226] Optionally, the device further comprises:

[0227] The neighboring difference determination module (not shown in the figure) is used to determine the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels; the target screening conditions corresponding to different motion states have different requirements for the neighboring difference of noise interference pixels relative to their neighboring pixels.

[0228] Optionally, the motion state includes at least two of the following motion states: a high-speed motion state, a low-speed motion state, and a stationary state;

[0229] The target screening condition includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the reference frame thermal image represent different targets, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the reference frame thermal image is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0230] The target screening conditions corresponding to different motion states have different requirements for the proximity difference of noise interference pixels relative to their neighboring pixels, including: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state.

[0231] Optionally, the target screening condition corresponding to the high-speed motion state further includes: if the grayscale of a pixel in the thermal image to be corrected and the grayscale of a pixel at a corresponding position in the thermal image of the reference frame represent the same target, and the difference between the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position in the thermal image of the reference frame is less than a preset difference threshold, then the pixel at the corresponding position in the thermal image to be corrected is determined as a noise interference pixel;

[0232] In the target screening condition corresponding to the high-speed motion state, the preset difference threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset difference threshold corresponding to the same target.

[0233] Optionally, the target screening condition further includes: the sum of the neighboring difference of the pixel in the thermal image to be corrected and the neighboring difference of the pixel at the corresponding position of the reference frame thermal image is less than a preset sum value threshold;

[0234] The target screening conditions corresponding to different motion states have different requirements for the proximity difference of noise interference pixels relative to their neighboring pixels, including: the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the stationary state.

[0235] Optionally, the target screening condition corresponding to the high-speed motion state further includes:

[0236] In the target screening condition corresponding to the high-speed motion state, the preset sum value threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset sum value threshold corresponding to the same target.

[0237] Optionally, the image correction module 903 includes:

[0238] A difference determination submodule (not shown in the figure) is used to determine, for each pixel in the thermal image to be corrected that has a noise interference pixel in its neighborhood, a neighborhood difference between the pixel and the noise interference pixel in its neighborhood;

[0239] A pixel determination submodule (not shown in the figure) is used to determine that the noise interference pixel is a pixel to be corrected if the neighboring difference satisfies a preset difference condition; wherein the preset difference condition indicates that the pixel and the noise interference pixel in its neighborhood correspond to the same target, and different motion states correspond to different preset difference conditions;

[0240] A correction parameter determination submodule (not shown in the figure), used to determine the correction parameter corresponding to the pixel to be corrected based on the neighboring difference;

[0241] The image correction submodule (not shown in the figure) is used to correct the pixels to be corrected according to the correction parameters to obtain a corrected thermal image.

[0242] Optionally, the preset difference condition includes that the adjacent difference is not greater than a preset difference threshold, and the preset difference conditions corresponding to different motion states include: the preset difference threshold corresponding to the high-speed motion state is greater than the preset difference threshold corresponding to the low-speed motion state, and / or the preset difference threshold corresponding to the low-speed motion state is greater than the preset difference threshold corresponding to the stationary state.

[0243] Optionally, the correction parameter determination submodule is specifically used to determine the correction parameter corresponding to the pixel to be corrected according to the neighboring difference and the preset correction degree parameter corresponding to the motion state; wherein the preset correction degree parameters corresponding to different motion states are different.

[0244] Optionally, the different preset correction degree parameters corresponding to the different motion states include: the value of the preset correction degree parameter corresponding to the high-speed motion state is greater than the value of the preset correction degree parameter corresponding to the low-speed motion state, and / or the value of the preset correction degree parameter corresponding to the low-speed motion state is greater than the value of the preset correction degree parameter corresponding to the stationary state.

[0245] Optionally, the motion state determination module 901 is specifically used to obtain multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected, wherein each group of angular velocities includes at least one of a swing angular velocity, a tilt angular velocity and a flip angular velocity; calculate the range and the sum between each type of angular velocity; and determine, based on the range and the sum, whether the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state or a stationary state.

[0246] Optionally, the motion state determination module 901 is specifically used to determine the motion state of the thermal imaging device as a high-speed motion state if the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold; if the range is less than the first preset range threshold and the sum is less than the first preset sum threshold, and when the range is greater than a second preset range threshold and the sum is greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a low-speed motion state; wherein the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold; if the range is not greater than the second preset range threshold and the sum is not greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a stationary state.

[0247] Optionally, the thermal image to be corrected is a target thermal image to be processed, or the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image.

[0248] Optionally, the device further comprises:

[0249] A preliminary correction module (not shown in the figure) is used to obtain the accumulated correction parameters obtained by accumulating the correction parameters corresponding to each frame of thermal image before the target thermal image; based on the accumulated correction parameters, the target thermal image is preliminarily corrected to obtain the preliminary corrected thermal image.

[0250] The embodiment of the present invention further provides an electronic device, such as Fig.10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0251] Memory 1003, used for storing computer programs;

[0252] The processor 1001 is used to implement the steps of the thermal imaging image correction method described in any of the above embodiments when executing the program stored in the memory 1003.

[0253] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0254] The communication interface is used for communication between the above electronic device and other devices.

[0255] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0256] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0257] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned thermal imaging image correction methods are implemented.

[0258] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any one of the thermal imaging image correction methods in the above embodiments.

[0259] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0260] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0261] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, computer-readable storage medium, and computer program product embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0262] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A thermal imaging image correction method, characterized in that: include: Determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected, wherein the motion state includes at least two of the following motion states: a high-speed motion state, a low-speed motion state, and a stationary state; Determining the neighborhood difference of each pixel in the thermal image to be corrected relative to its neighboring pixels; Determine noise interference pixels from the thermal image to be corrected using the target screening condition corresponding to the motion state; wherein the target screening condition includes: the sum of the neighboring differences of each pixel in the thermal image to be corrected and the neighboring differences of the pixels at the corresponding position of the reference frame thermal image is less than a preset sum value threshold, the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the static state; The thermal image to be corrected is corrected according to the noise interference pixels.

2. The method according to claim 1, characterized in that The target screening conditions corresponding to the high-speed motion state also include: In the target screening condition corresponding to the high-speed motion state, the preset sum value threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset sum value threshold corresponding to the same target.

3. The method according to claim 1, characterized in that The step of determining the motion state of the thermal imaging device corresponding to the thermal image to be corrected includes: Acquire multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected, wherein each group of angular velocities includes at least one of a swing angular velocity, a tilt angular velocity, and a flip angular velocity; Calculate the range and sum of angular velocities for each type; Based on the range and the sum, it is determined that the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state or a stationary state.

4. The method according to claim 3, characterized in that The determining, based on the range difference and the sum value, that the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state, or a stationary state includes: If the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold, determining the motion state of the thermal imaging device as a high-speed motion state; If the range is less than a first preset range threshold and the sum is less than a first preset sum threshold, and when the range is greater than a second preset range threshold and the sum is greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a low-speed motion state; wherein the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold; If the range is not greater than a second preset range threshold and the sum is not greater than a second preset sum threshold, the motion state of the thermal imaging device is determined to be a stationary state.

5. The method according to any one of claims 1 to 4, characterized in that: The thermal image to be corrected is a target thermal image to be processed, or the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image.

6. The method according to claim 5, characterized in that The method further comprises: Acquire an accumulated correction parameter obtained by accumulating correction parameters corresponding to each frame of thermal image before the target thermal image; The target thermal image is preliminarily corrected based on the accumulated correction parameters to obtain the preliminarily corrected thermal image.

7. A thermal imaging image correction device, characterized in that: include: A motion state determination module, used to determine the motion state of the thermal imaging device corresponding to the thermal image to be corrected, wherein the motion state includes at least two of the following motion states: a high-speed motion state, a low-speed motion state, and a stationary state; A neighboring difference determination module, used to determine the neighboring difference of each pixel in the thermal image to be corrected relative to its neighboring pixels; An interference pixel determination module is used to determine noise interference pixels from the thermal image to be corrected by using the target screening conditions corresponding to the motion state; wherein the target screening conditions include: the sum of the neighboring differences of each pixel in the thermal image to be corrected and the neighboring differences of the pixels at the corresponding positions of the reference frame thermal image is less than a preset sum value threshold, the preset sum value threshold corresponding to the high-speed motion state is greater than the preset sum value threshold corresponding to the low-speed motion state, and / or the preset sum value threshold corresponding to the low-speed motion state is greater than the preset sum value threshold corresponding to the static state; An image correction module is used to correct the thermal image to be corrected according to the noise interference pixels.

8. The device according to claim 7, characterized in that The target screening conditions corresponding to the high-speed motion state also include: In the high-speed motion state corresponding target screening condition, the preset sum value threshold corresponding to the pixel grayscale in the thermal image to be corrected and the pixel grayscale at the corresponding position of the reference frame thermal image when representing different targets is greater than the preset sum value threshold corresponding to the same target; The motion state determination module is specifically used to obtain multiple groups of angular velocities of the thermal imaging device corresponding to the thermal image to be corrected, wherein each group of angular velocities includes at least one of a swing angular velocity, a tilt angular velocity, and a flip angular velocity; calculate the range and sum between each type of angular velocity; and determine, based on the range and the sum, whether the motion state of the thermal imaging device is a high-speed motion state, a low-speed motion state, or a stationary state; The motion state determination module is specifically used to determine the motion state of the thermal imaging device as a high-speed motion state if the range is not less than a first preset range threshold and the sum is not less than a first preset sum threshold; if the range is less than the first preset range threshold and the sum is less than the first preset sum threshold, and when the range is greater than a second preset range threshold and the sum is greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a low-speed motion state; wherein the first preset range threshold is greater than the second preset range threshold, and the first preset sum threshold is greater than the second preset sum threshold; if the range is not greater than the second preset range threshold and the sum is not greater than the second preset sum threshold, determine the motion state of the thermal imaging device as a stationary state; The thermal image to be corrected is a target thermal image to be processed, or the thermal image to be corrected is a preliminary corrected thermal image obtained by performing preliminary correction on the target thermal image; The device also includes: The preliminary correction module is used to obtain the accumulated correction parameters obtained by accumulating the correction parameters corresponding to each frame of thermal image before the target thermal image; based on the accumulated correction parameters, the target thermal image is preliminarily corrected to obtain the preliminary corrected thermal image.

Images