An Infrared Moving Blurred Target Recognition Method Based on Temperature Feature Sequence

Abstract

The present invention discloses an infrared moving blurred target recognition method based on a temperature feature sequence, which specifically relates to the technical field of infrared target recognition. With the temperature feature sequence as the core, by studying the distinguishing features of the temperature feature sequences of the true position of the target and the blurred area, the Yolov5 network is used to detect the target and background interference. The temperature feature sequence is extracted by using a temperature field model, a projected area model, and a triangular facet analysis method. A temperature feature sequence model of the target and the background is established, and an improved LSTM network is used to discriminate the target and background interference, so as to realize the separation of the target and the background and achieve high-precision positioning of the blurred target.

Description

Technical Field

[0001] The present invention relates to the technical field of infrared target recognition, and particularly relates to an infrared moving blurred target recognition method based on a temperature feature sequence. Background Art

[0002] Infrared imaging guidance has the advantage of strong anti-interference ability and has great advantages in environments with large scenes and a lot of clutter. It is one of the most widely used guidance technologies at present. Compared with expensive precision-guided missiles, low-cost precision-guided missiles are small in size, simple in structure, and low in price, and are suitable for a variety of scenarios with a very wide range of applications. Uncooled strapdown detectors can meet the cost requirements of low-cost precision-guided missiles. In recent years, with the development of materials and technologies, cooled strapdown infrared focal plane detectors made of materials such as HgCdTe and InSb have been widely used in high-end military fields; uncooled strapdown infrared focal plane detectors made of materials such as VOx and α-Si have quickly occupied the civilian market due to their advantages in price and volume.

[0003] However, due to factors such as the relative movement between the imaging device and the target and the device jitter during the integration time of the uncooled strapdown detector, it is easy to cause image information loss and quality degradation, resulting in various degrees of motion blur in the imaging of the uncooled strapdown detector, making it difficult to effectively utilize the imaging information and bringing difficulties to target detection. At present, extensive research has been carried out on the problem of global motion blur in images, and phased results have been achieved.

[0004] However, the degree of motion blur that the image de-motion blur technology can effectively remove is currently relatively limited, and it is difficult to process some severely motion-blurred images. At the same time, the algorithm model combining the image de-blurring algorithm and the target detection algorithm is usually large in size, difficult to deploy, and poor in real-time performance, and it is difficult to meet the requirements of small and independently integrated imaging detection devices for eliminating the influence of motion blur.

[0005] Therefore, overcoming the motion blur problem of uncooled strapdown infrared imaging and the complex problem of temperature signal background clutter in large field-of-view scenes, and realizing high-precision infrared moving blurred target positioning can greatly improve the target detection performance of small and integrated imaging detection devices such as micro missiles and unmanned aerial vehicles, expand their scope of application, and reduce the deployment cost and hardware cost, which has important significance. Summary of the Invention

[0006] For this reason, the present invention provides an infrared moving blurred target recognition method based on a temperature feature sequence to solve the problems proposed in the background art.

[0007] To achieve the above object, the present invention provides the following technical solution: An infrared moving blurred target recognition method based on a temperature feature sequence, comprising the following steps:

[0008] S1. Use a drone, an infrared thermal imager, and a turntable to capture an infrared blurred dataset;

[0009] S2. Based on the temperature waveform sequence discrimination features of the target's true position and the blurred area, establish a temperature field model for the target and background interference, establish a connection between the gray value and the radiation signal received by the infrared detector, and indirectly extract the temperature waveform from the gray distribution of the image by studying the relationship between the radiation signal and the temperature of the target and background;

[0010] Establish a projection area model for the target, including:

[0011] Geometric projection area: The static projection area based on the geometric shape of the drone and the line-of-sight angle;

[0012] Dynamic attitude correction: Incorporate the dynamic influence of attitude changes on the projection area;

[0013] Jitter blur expansion: The expansion effect of image blur caused by projectile jitter on the projection area;

[0014] S3. Input the infrared blurred image dataset obtained in S1 into the YOLOv5 network for rough target detection. The rough target detection simultaneously detects the target and background interference. Discretize the radiation on the target surface by the triangular facet method, combine the pixel gray value of the image with the projection area of the facet, extract the gray values of the target and background regions, calculate the total gray value, solve the radiation values of the target and background interference regions, and form the temperature feature sequences of the target and background interference according to the temperature field model and projection area model described in S2;

[0015] S4. Combine the temperature feature sequences of the target and background interference described in S3 with the temperature field model and projection area model in S2, and use the improved LSTM time-series deep learning algorithm to perform high-precision detection and recognition of infrared maneuvering targets, accurately locate the targets in the infrared blurred images, and achieve the separation of the targets and the background.

[0016] Preferably, the infrared detector is a non-cooled strap-down detector.

[0017] Preferably, the temperature field model described in S2 is constructed as follows:

[0018] (1) The surface temperature of the target and its radiance satisfy the following relationship:

[0019] ;

[0020] is the spectral emissivity of the target surface, is the Stefan-Boltzmann constant, is the target surface temperature distribution. The radiation signal received by the infrared detector is converted into the gray value of the image through the spectral response function:

[0021] ;

[0022] Among them, K is the response coefficient of the infrared detector, is the working band range of the infrared detector, are the Planck constant, the speed of light and the Boltzmann constant respectively, is the spectral response function. Since the gray value output by the detector is the integration result of the radiation signal, the relationship between the gray value and the temperature field is a non-linear mapping.

[0023] (2) In the imaging background of the infrared detector, the connection between the target temperature field and the image gray value is significantly affected by the characteristics of the infrared detector and motion jitter. Especially because the uncooled strapdown detector has a long integration time, during the movement and jitter of the target, the radiation signal will spread to different pixel points, resulting in the radiation information reflected by the gray value being the superposition of multiple time segments. Therefore, a dynamic temperature field expression method is constructed based on the fuzzy image gray value and the characteristics of the uncooled strapdown detector. By establishing the mapping relationship between the temperature field, radiation brightness and gray value, the long integration time of the infrared detector and the motion blur effect of the target are comprehensively considered, and the target temperature field is accurately reconstructed and dynamically described.

[0024] The integration time of the uncooled strapdown detector is relatively long (usually from several milliseconds to dozens of milliseconds), and its imaging principle results in that the radiation signal received by each pixel point is the cumulative value of the radiation energy of the target within this time range:

[0025] ;

[0026] Among them: is the gray value of the pixel point; K is the response coefficient; is the integration time; is the radiation brightness of the target at time ; represents the moment when the radiation energy starts to accumulate; when the target jitters, the radiation signal of the target will not always be transmitted to the same pixel point, resulting in a blur effect in the gray value of a single pixel point.

[0027] During the jitter process, the radiation energy of the target temperature field spreads to multiple pixel points, and the result is manifested as the fuzzy superposition of gray values on the image. The gray value distribution is expressed as:

[0028] ;

[0029] is the fuzzy weight, reflecting the signal diffusion distribution caused by jitter; represents the radiance after diffusion; the diffusion characteristics of the jitter on the radiation signal will further weaken the direct relationship between the gray value and the temperature field, making the gray value the superposition result of the radiation signal affected by the integration time and jitter.

[0030] Preferably, for the projection area model described in S3, the projection area is the plane projection area of all surfaces of the target perpendicular to the observation angle within the field of view at a certain observation angle. During the calculation of the infrared target radiation characteristics, the target size will affect the gray value of the imaging point, and the shape characteristics will be reflected in the change trend of the projection area in the image. It comprehensively considers three factors: geometric projection area: the static projection area based on the geometric shape of the UAV and the line-of-sight angle; dynamic attitude correction: adding the dynamic influence of attitude changes on the projection area; jitter blur expansion: considering the expansion effect of image blur caused by the projectile jitter on the projection area. The specific composition of the projection area model is as follows:

[0031] The overall calculation of the UAV projection area is divided into three parts:

[0032] ;

[0033] is the static projection area, determined by the geometric configuration of the UAV; is the dynamic projection area, caused by attitude changes; is the blur expansion area, caused by the projectile jitter; the projection area model is divided into two types, namely the fixed-wing UAV projection area model and the rotary-wing UAV projection area model;

[0034] (1) Fixed-wing UAV projection area model:

[0035] The geometric model of a fixed-wing UAV can be simplified to a polyhedron structure with a front and rear cone and two side wings. The front cone (nose) and the rear cone (tail) are represented by trapezoidal and triangular faces, and the main wing is rectangular. The main parameters include the fuselage length: L, which is the axial superposition of the front cone and the rear cone; the wingspan: 2R, symmetric left and right, that is, the total width is 2R; the wing thickness: h, indicating the relative thinness of the wing; the tail wing width: a, the tail wing is relatively narrow and simplified to a rectangle.

[0036] The projection area model of a fixed-wing UAV consists of the following three parts: static projection area: the basic geometric projection area composed of the fuselage, wings, and tail wings. Dynamic correction term: the dynamic change of the projection area caused by attitude changes (pitch, roll, yaw). Blur expansion term: the expansion of the projection area caused by jitter and the influence of the infrared detector resolution.

[0037] The static projected area includes the following parts:

[0038] Fuselage projected area:

[0039] ;

[0040] L is the length of the fuselage, W is the width of the fuselage, and H is the thickness of the fuselage, which reflects the vertical dimension of the fuselage. is the pitch angle, which is the angle of rotation of the UAV around the transverse axis. In the calculation of the projected area, the pitch angle affects the size and shape of the fuselage's projection on the projection plane.

[0041] Wing projected area:

[0042] ;

[0043] is the roll angle, that is, the angle of rotation of the UAV around the longitudinal axis.

[0044] Tail wing projected area:

[0045] ;

[0046] is the yaw angle, that is, the angle of rotation of the UAV around the vertical axis (usually the vertical axis of the UAV).

[0047] Total static projected area:

[0048] ;

[0049] The dynamic correction term is caused by attitude changes and is corrected by changes in the pitch angle, roll angle, and yaw angle.

[0050] ;

[0051] Among them, the attitude change can be described by a rotation matrix and the projected area is corrected in real time.

[0052] The fuzzy extended area is caused by jitter and the resolution of the infrared detector:

[0053] When the jitter range is an ellipse, the fuzzy area is:

[0054] ;

[0055] is the offset on the x-axis, is the offset on the y-axis.

[0056] Calculated using the Gaussian blur weight function:

[0057] ;

[0058] The standard deviation, which is an important parameter of the Gaussian distribution, is used to control the degree of blurring.

[0059] Total blurred expansion area:

[0060] ;

[0061] Combining the static projection area, the dynamic correction term, and the blurred expansion term, the total projection area model of the fixed-wing UAV is:

[0062] ;

[0063] (2) Projection area model of the rotary-wing UAV:

[0064] The rotary-wing UAV can be simplified into a flat square central body, with four symmetrically distributed arms connected to the rotors, and circular rotors at the ends of the arms. Each rotor area is simplified into a circle, and the arms are slender rectangles. The main parameters include: Central body size: side length a of the square; Arm length: l, extending from the edge of the central body to the center of the rotor; Rotor radius: r, the rotor is simplified into a circle; Arm width: b, the cross-section of the arm is a rectangle.

[0065] The projection area model of the quadrotor UAV consists of the following three parts: Static projection area: the basic geometric projection areas of the central body and the rotors. Dynamic correction term: the change in the dynamic projection area caused by attitude changes (pitch, roll, yaw) and the periodic motion of the rotors. Blurred expansion term: the expansion of the projection area caused by jitter and the influence of the infrared detector resolution.

[0066] The static projection area includes the following parts:

[0067] Central body projection area:

[0068]

[0069] is the pitch angle, which is the angle of the UAV rotating around the transverse axis.

[0070] Rotor projection area (each rotor):

[0071]

[0072] is the roll angle, that is, the angle of the UAV rotating around the longitudinal axis.

[0073] Total static projection area:

[0074]

[0075] The dynamic correction term is caused by attitude changes and the periodic motion of the rotors:

[0076]

[0077] where is the dynamic projected area of the \(i\)th rotor, considering the time factor.

[0078] The fuzzy extended area is caused by jitter and the resolution of the infrared detector:

[0079] When the jitter range is circular, the fuzzy area is:

[0080]

[0081] Calculated using the Gaussian blur weight function:

[0082]

[0083] Total fuzzy extended area:

[0084]

[0085] Combining the static projected area, dynamic correction term, and fuzzy extension term, the total projected area model of the quadrotor UAV is:

[0086]

[0087] (3) Calculate the projected area of the fuzzy region using the projected area model

[0088] The motion-blurred region is jointly affected by the target motion trajectory and the sensor integration time, and its projected area can be calculated through the fuzzy weight and the image pixel values:

[0089]

[0090] is the pixel set of the fuzzy region, is the actual physical area of a single pixel. The fuzzy weight can be approximated as a Gaussian distribution, so as to extract the fuzzy weight from the image.

[0091]

[0092] is the center point of the target's true region. \(\sigma\) represents the degree of fuzziness, which can be obtained by fitting the gradient change between the fuzzy image and the non-fuzzy image.

[0093] The target's true projected area is directly calculated from the pixel values of the infrared image:

[0094] ;

[0095] A set of pixels for the blurred area.

[0096] Quantify the degree of blurring through the projected areas of the blurred area and the real area:

[0097]

[0098] By the triangular element method, according to the established temperature field model and projected area model, combined with the gray value of the image and the projected area of the element, solve the radiation value of the target and background interference during the movement process, so as to establish a temperature characteristic sequence model of the target and background interference.

[0099] Preferably, the improved LSTM time series deep learning algorithm is specifically composed as follows:

[0100] (1) A one-dimensional convolutional neural network is used to process the original signal, and the pooling layer is replaced by a strided convolutional layer to retain the original signal information; the softmax classifier of the traditional convolutional neural network is replaced by an SVM classifier;

[0101] (2) Select a bidirectional long short-term memory recurrent neural network Bi-LSTM, which includes two LSTM layers in the forward and reverse directions and has a stronger ability to extract long sequence features.

[0102] The present invention also discloses that the above infrared moving blurred target recognition method based on the temperature characteristic sequence is applied to detect and recognize different types of targets, including unmanned aerial vehicles and unmanned ships.

[0103] The present invention has the following advantages:

[0104] The present invention takes the temperature characteristic sequence as the core, distinguishes the characteristics of the temperature characteristic sequences of the real position of the target and the blurred area through research, uses the Yolov5 network to detect the target and background interference, extracts the temperature characteristic sequence by using the temperature field model, projected area model and triangular element analysis method, establishes a temperature characteristic sequence model of the target and background, and uses the improved LSTM network to distinguish the target and background interference, so as to realize the separation of the target and background, and realize the high-precision positioning of the blurred target. Compared with the prior art, it prevents the loss of image information and quality degradation caused by factors such as the relative movement between the imaging device and the target and the device jitter during the integration time of the infrared detector, avoids various degrees of motion blur in the imaging of the uncooled strapdown detector, and effectively utilizes the imaging information for target detection. Description of the Drawings

[0105] Figure 1 It is a schematic diagram of the steps of the infrared moving blurred target recognition method based on the temperature characteristic sequence according to the present invention;

[0106] Figure 2 Schematic diagram of establishing a temperature field model, a projection area model, and extracting a temperature feature sequence by the triangular facet analysis method according to the present invention;

[0107] Figure 3 Temperature feature sequence diagram according to the present invention. Specific implementation manners

[0108] The following specific embodiments illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0109] The embodiments of the present invention select the computer configuration as follows: i5-12400F core processor, RTX 3060Ti graphics processor, main frequency 2.50 GHz, memory 32GB, and the operating system is windows 10.

[0110] An infrared moving blurred target recognition method based on a temperature feature sequence according to the present invention is implemented based on the Pytorch 1.7.1 deep learning framework toolkit.

[0111] As Figures 1 to 2 shown, an infrared moving blurred target recognition method based on a temperature feature sequence specifically includes the following steps:

[0112] Step 1: Establish an infrared blurred image data set, and the specific steps are as follows:

[0113] Mount an infrared thermal imager on a turntable, set the jitter amplitude and jitter frequency of the turntable through the turntable control terminal, use buildings, the sky, trees, etc. as the background, and use an unmanned aerial vehicle as the target to capture an infrared blurred data set with different degrees of blur.

[0114] Step 2: Establish a temperature feature sequence model for the target and background interference, and the specific steps are as follows:

[0115] Study the distinguishing features of the temperature waveform sequences between the true position of the research target and the blurred area, establish a temperature field model, establish a connection between the gray value and the radiation signal received by the infrared detector, and indirectly extract the target temperature waveform from the gray distribution of the image by studying the relationship between the radiation signal and the temperature field; In the imaging background of the uncooled strapdown detector, due to the long integration time of the uncooled strapdown detector, during the movement and jitter of the target, the radiation signal will spread to different pixel points, resulting in the radiation information reflected by the gray value being the superposition of multiple time segments. Based on the gray value of the blurred image and the characteristics of the uncooled infrared detector, a dynamic temperature field expression method is constructed, comprehensively considering the long integration time of the uncooled strapdown detector and the motion blur effect of the target, and studying the accurate reconstruction and dynamic description of the target temperature field.

[0116] Establish a projection area model. The projection area is the plane projection area of all surfaces of the target within the field of view perpendicular to the observation angle at a certain observation angle. During the calculation of the infrared target radiation characteristics, the target size will affect the gray value of the imaging point, and the shape characteristics will be reflected in the change trend of the projection area in the image.

[0117] Comprehensively consider the following points:

[0118] Geometric projection area: The static projection area based on the geometric shape of the UAV and the line of sight angle.

[0119] Dynamic attitude correction: Add the dynamic influence of attitude changes on the projection area.

[0120] Jitter blur expansion: Consider the expansion effect of image blur caused by projectile jitter on the projection area, which is affected by three factors.

[0121] Step 3: Extract the temperature feature sequences of the target and the background. The specific steps are as follows:

[0122] Use the Yolov5 network to perform rough detection on the infrared blurred dataset. The rough detection can detect both the target and background interference at the same time. Using the triangular facet method, combined with the gray value of the image and the projection area of the facet, solve the radiation values of the target and background regions to form the temperature feature sequences of the two. The sequence is shown as Figure 3 shown.

[0123] Extract the temperature feature sequences of the target and the background blurred area of the consecutive frame images frame by frame. The temperature feature sequence of the target is the positive sample (blue), and the temperature feature sequence of the blurred area is the negative sample (orange).

[0124] Step 4: Perform high-precision recognition and positioning of the target. The specific steps are as follows:

[0125] Combine the temperature feature sequence model described in step 2 and the temperature feature sequences of the target and the background described in step 3, and output the recognition result of the temperature feature sequence through an improved LSTM time series deep learning algorithm, so as to achieve the separation of the target and the background and achieve high-precision positioning of the target.

[0126] The present invention takes the temperature feature sequence as the core, studies the distinguishing features of the temperature feature sequences of the true position of the target and the fuzzy area, uses the Yolov5 network to detect the target and background interference, extracts the temperature feature sequence by using the temperature field model, the projection area model and the triangular facet analysis method, establishes the temperature feature sequence model of the target and the background, and uses the improved LSTM network to discriminate the target and background interference, so as to achieve the separation of the target and the background and achieve high-precision positioning of the fuzzy target.

[0127] The above-described invention can also be extended to other similar infrared target detection tasks. Just set appropriate parameters and retrain the model using a simulation dataset or a real dataset to obtain an ideal model.

[0128] Although the present invention has been described in detail above with general descriptions and specific embodiments, on the basis of the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. An infrared moving blurred target recognition method based on temperature feature sequences, comprising the following steps: S1. Use a drone, an infrared detector, and a turntable to capture an infrared blurred dataset; The infrared detector is an uncooled strapdown detector; characterized in that: S2. Based on the temperature waveform sequence discrimination features of the target's true position and the blurred area, establish a temperature field model of the target and background interference, establish a connection between the gray value and the radiation signal received by the infrared detector, and indirectly extract the temperature waveform from the gray distribution of the image by studying the relationship between the radiation signal and the target and background temperatures; The temperature field model is constructed as follows: (1) The surface temperature T(x, y) of the target and its radiance L(x, y) satisfy the following relationship: L(x,y) = ò(x,y)·σ·T(x,y) 4 ; ò(x, y) is the spectral emissivity of the target surface, σ is the Stefan-Boltzmann constant, T(x, y) is the surface temperature distribution of the target, and the radiation signal received by the infrared detector is converted into the gray value of the image through the spectral response function: where K is the response coefficient of the infrared detector, [λ1, λ2] is the working band range of the infrared detector, h, c, and k B are the Planck constant, the speed of light, and the Boltzmann constant respectively, and R(λ) is the spectral response function; (2) Under the imaging background of the infrared detector, the radiation signal received by each pixel is the cumulative value of the radiation energy of the target within this time range: Where: G1(x, y) is the gray value of the pixel; K is the response coefficient; τ is the integration time; L(x(t), y(t), t) is the radiance of the target at time t; t0 represents the moment when the radiation energy starts to accumulate; when the target jitters, the radiation energy of the target temperature field diffuses to multiple pixels, and the result is manifested as the blurred superposition of gray values on the image, and the gray value distribution is expressed as: G'(x, y) = ∫L(x', y')·W(x - x', y - y')dA W(x - x', y - y') is the blur weight, reflecting the signal diffusion distribution caused by jitter; L(x', y') represents the radiance after diffusion; Establish a projection area model of the target, including: Geometric projection area: The static projection area based on the geometric shape of the drone and the line-of-sight angle; Dynamic attitude correction: Add the dynamic influence of attitude changes on the projection area; Jitter blur expansion: The expansion effect of image blur caused by projectile jitter on the projection area; The projection area model is specifically composed as follows: The overall calculation of the drone projection area is divided into three parts: A final A(t) = A static A(t) + A dynamic A(t) + A blur A(t); A static (t) is the static projected area, which is determined by the geometric configuration of the UAV; A dynamic (t) is the dynamic projected area, which is caused by attitude changes; A blur (t) is the fuzzy expansion area, which is caused by the projectile body jitter; S3. Input the infrared blurred image dataset obtained in S1 into the YOLOv5 network for rough target detection. The rough target detection simultaneously detects the target and background interference. Discretize the surface radiation of the target by the triangular facet method, combine the pixel gray value of the image with the projection area of the facet, extract the gray values of the target and background regions, calculate the total gray value, solve the radiation values of the target and background interference regions, and form the temperature feature sequences of the target and background interference according to the temperature field model and projection area model described in S2; S4. Combine the temperature feature sequences of the target and background interference described in S3 with the temperature field model and projection area model in S2, and use the improved LSTM time series deep learning algorithm to perform high-precision detection and recognition of infrared maneuvering targets, accurately locate the targets in the infrared blurred images, and realize the separation of the target and the background.

Images