Infrared focal plane array attitude estimation method and device

Abstract

The present invention discloses a method and device for estimating the attitude of an infrared focal plane array (IFPA), belonging to the technical field of rotating carrier attitude estimation. The method comprises: extracting features of IFPA data using a sliding window to obtain feature vectors of multiple windows; selecting a block partitioning method using an MLP decision network to divide the IFPA data within the window into multiple sub-blocks; calling a corresponding distributed LSTM model from a distributed LSTM model pool based on the IFPA data and block partitioning method of each sub-block to estimate the attitude angle sine and cosine values ​​of each sub-block; fusing the attitude angle sine and cosine values ​​of each sub-block based on a self-attention mechanism, and converting the fused attitude angle sine and cosine values ​​into attitude angles in an angle system to obtain the IFPA attitude estimation result. The present invention improves the accuracy and real-time performance of attitude angle estimation and can be widely applied to attitude estimation of high-speed rotating carriers.

Description

Technical Field

[0001] The invention relates to an infrared focal plane array attitude estimation method and device, and belongs to the technical field of rotating carrier attitude estimation. Background Art

[0002] In addition to geomagnetic, inertial, satellite and other attitude measurement methods, the study of the Earth's infrared radiation characteristics has received increasing attention in recent years. The method of using the Earth's infrared radiation field to measure the attitude of rotating carriers has the advantages of low cost, good performance, and all-weather.

[0003] Currently, there is a wide range of research on infrared-based methods for measuring vehicle attitude parameters. Existing infrared focal plane array attitude calculation methods include direct calculation, traditional filter estimation, and infrared image-based methods. However, direct calculation methods are susceptible to noise and often suffer from large calculation errors. Traditional filter estimation methods, such as the Kalman filter and its extended forms, rely heavily on the accuracy of the vehicle motion model for calculation accuracy. Infrared image-based attitude measurement methods are highly dependent on the quality of the infrared image and the accuracy of feature extraction. Furthermore, the inherent non-uniform noise of the infrared focal plane array can interfere with key features in the image, such as target edges and corners, thereby affecting calculation accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide an infrared focal plane array attitude estimation method and device, which performs carrier infrared attitude estimation through adaptive blocking, distributed LSTM model and self-attention mechanism dynamic weight fusion technology, improves the accuracy and real-time performance of attitude angle estimation, and can be widely used in attitude estimation of high-speed rotating carriers.

[0005] In order to achieve the above objectives / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] In one aspect, the present invention provides an infrared focal plane array attitude estimation method, comprising:

[0007] The features of infrared focal plane array data are extracted using sliding windows to obtain feature vectors of multiple windows;

[0008] According to the feature vector of each window, the MLP decision network is used to select the block method corresponding to the window, and the infrared focal plane array data in the window is divided into multiple sub-blocks;

[0009] According to the infrared focal plane array data and block division method of each sub-block, the corresponding distributed LSTM model is called from the distributed LSTM model pool to estimate the sine and cosine values ​​of the attitude angle of each sub-block;

[0010] The attitude angle sine and cosine estimation values ​​of each sub-block are fused based on the self-attention mechanism, and the fused attitude angle sine and cosine estimation values ​​are converted into attitude angle estimation values ​​in the angle system to obtain the attitude estimation result of the infrared focal plane array.

[0011] In combination with the first aspect, further, extracting features of infrared focal plane array data using a sliding window to obtain feature vectors of multiple windows includes:

[0012] Using a sliding window to divide the infrared focal plane array data into multiple window data;

[0013] Each window data is composed of a A three-dimensional tensor of , where Indicates the number of rows of the infrared focal plane array sensor, represents the number of columns of the infrared focal plane array sensor, and C represents the window length;

[0014] Calculating statistical features of the three-dimensional tensor data of each window, wherein the statistical features include overall mean, overall standard deviation, range, frame-by-frame mean sequence, and frame-by-frame standard deviation sequence;

[0015] Concatenate the statistical features of each window to form a one-dimensional feature vector of the window data .

[0016] Combined with the first aspect, a sliding window will be used to extract the features of historical infrared focal plane array data to obtain historical feature vectors of multiple windows. The historical feature vectors of different windows will be used as the input of the MLP decision network, and the probabilities of different blocking methods will be used as the output of the MLP decision network. The attitude angle estimation errors under different blocking methods will be used as blocking effect labels to train the MLP decision network to obtain a trained MLP decision network.

[0017] Combined with the first aspect, further, for the historical feature vector of each window, the block method with the highest probability output by the MLP decision network is selected as the optimal block method for the window;

[0018] Dividing the infrared focal plane array data in the window into several sub-blocks according to the optimal block division method;

[0019] Assume that the optimal block partitioning method is ,in, 、 Respectively represent the number of rows and columns of each sub-block under the optimal blocking method, and each frame of infrared focal plane array data in the window is expressed as follows after blocking processing:

[0020] ;

[0021] in, represents the matrix after block processing, Represents the sub-block at the mth row and nth column after block processing, , , , A and B represent the number of rows and columns of the infrared focal plane array sensor, respectively;

[0022] is a The matrix expression is as follows:

[0023] ;

[0024] in, Indicates the first Row, No. The value of the column, , .

[0025] In combination with the first aspect, further, calling the corresponding distributed LSTM model from the distributed LSTM model pool based on the infrared focal plane array data and the block division method of each sub-block to estimate the attitude angle sine and cosine values ​​of each sub-block includes:

[0026] Select the distributed LSTM model corresponding to the block mode from the distributed LSTM model pool;

[0027] Convert the infrared focal plane array data of each sub-block into model input data according to the input format of the distributed LSTM model;

[0028] The selected distributed LSTM model is used to process the model input data to obtain the attitude angle sine and cosine estimation values ​​of each sub-block.

[0029] In combination with the first aspect, further, the infrared focal plane array data of each sub-block is converted into model input data according to the input format of the distributed LSTM model, including:

[0030] Flatten each frame of infrared focal plane array data of each sub-block by row to obtain the feature vector of each frame:

[0031] ;

[0032] in, Represents the sub-block at row m and column n A frame feature vector of Represents a sub-block Middle Row, No. Columns of infrared focal plane array data, , , M and N are the total number of rows and columns of sub-blocks in the window, and T represents the transpose operation;

[0033] use Moment sub-block The feature vectors of the current frame and the historical C-1 frame are concatenated. The input data of the time-distributed LSTM model is expressed as follows:

[0034] ;

[0035] in, express The input data of the time-distributed LSTM model, express Moment sub-block The feature vector of the current frame in , C represents the window length.

[0036] In combination with the first aspect, further, each distributed LSTM model in the distributed LSTM model pool includes an input layer, two layers of LSTM networks with Dropout regularization, a fully connected layer and an output layer;

[0037] After the model input data enters the distributed LSTM model through the input layer, it passes through two LSTM networks with Dropout regularization in sequence. Then get the hidden state; fully connected layer Mapping hidden states to features; output layer Output the sine and cosine values ​​of the attitude angle of each sub-block according to the mapping features; the formula is as follows:

[0038] ;

[0039] ;

[0040] ;

[0041] in, express The sub-block at the mth row and nth column at the moment The corresponding model input data, express The corresponding hidden state, express The corresponding mapping features, Represents a sub-block The estimated sine of the pitch angle, Represents a sub-block The estimated value of the pitch angle cosine, Represents a sub-block The estimated value of the sine of the roll angle is Represents a sub-block The estimated value of the roll angle cosine is , , M and N are the total number of rows and columns of sub-blocks in the window respectively.

[0042] In combination with the first aspect, further, the attitude angle sine and cosine estimation values ​​of each sub-block are fused based on the self-attention mechanism, and the fused attitude angle sine and cosine estimation values ​​are converted into attitude angle estimation values ​​in an angle system to obtain an infrared focal plane array attitude estimation result, including:

[0043] Generate the block output matrix corresponding to the window according to the attitude angle sine and cosine estimation values ​​of each sub-block :

[0044] ;

[0045] in, Represents the sub-block at row m and column n The estimated sine and cosine values ​​of the attitude angle are vectors, Represents a sub-block The estimated sine of the pitch angle, Represents a sub-block The estimated value of the pitch angle cosine, Represents a sub-block The estimated value of the sine of the roll angle is Represents a sub-block The estimated value of the roll angle cosine is , , M and N are the total number of rows and columns of sub-blocks in the window respectively;

[0046] Using block output matrix And the self-attention mechanism calculates the similarity matrix between each sub-block in the window :

[0047] ;

[0048] in, Represents the attitude angle sine and cosine value vector of the sub-block at row m and column n and the attitude angle sine and cosine value vector of the sub-block in row k and column l The similarity between them, T represents the transposition operation;

[0049] The similarity matrix between each sub-block in the window conduct Normalize and get the attention weight matrix of each sub-block in the window :

[0050] ;

[0051] ;

[0052] in, Indicates the degree of reference to the sub-block information of the kth row and lth column when fusing the output results of the sub-block of the mth row and nth column;

[0053] Using the attention weight matrix Perform weighted summation on the attitude angle sine and cosine estimation values ​​of each sub-block in the window to obtain the fused attitude angle sine and cosine estimation value :

[0054] ;

[0055] in, It represents the fusion output vector of the sub-block of the mth row and nth column after weighted fusion of all sub-block information under the self-attention mechanism, and T represents the transpose operation;

[0056] According to the fused attitude angle sine and cosine values Perform reduction processing to obtain global fusion output :

[0057] ;

[0058] Global fusion output Perform two-norm normalization:

[0059] ;

[0060] in, represents the normalized fusion vector, 、 、 and They represent the estimated value of the pitch angle sine component, the estimated value of the pitch angle cosine component, the estimated value of the roll angle sine component, and the estimated value of the roll angle cosine component after normalization and fusion respectively;

[0061] Normalize the fusion vector Reverse encoding The attitude angle estimation value within the range is used as the attitude estimation result of the infrared focal plane array.

[0062] In a second aspect, the present invention provides an infrared focal plane array attitude estimation device, comprising:

[0063] A feature extraction module is used to extract features of infrared focal plane array data using a sliding window to obtain feature vectors of multiple windows;

[0064] A block division module is used to select a block division method corresponding to a window by using an MLP decision network according to the feature vectors of each window, and divide the infrared focal plane array data in the window into multiple sub-blocks;

[0065] The attitude angle estimation module is used to call the corresponding distributed LSTM model from the distributed LSTM model pool based on the infrared focal plane array data and block division method of each sub-block to estimate the attitude angle sine and cosine values ​​of each sub-block;

[0066] The attitude angle fusion conversion module is used to fuse the attitude angle sine and cosine estimation values ​​of each sub-block based on the self-attention mechanism, and convert the fused attitude angle sine and cosine estimation values ​​into attitude angle estimation values ​​in the angle system to obtain the infrared focal plane array attitude estimation result.

[0067] In a third aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the infrared focal plane array attitude estimation method described in the first aspect are implemented.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] The present invention proposes a method and device for infrared focal plane array attitude estimation. The method selects a block mode based on an MLP decision network, performs block processing on the time series data output by the infrared focal plane array sensor, and divides the data into multiple sub-blocks for synchronous processing, which can effectively improve the efficiency and real-time performance of the estimation. The present invention estimates the attitude angle sine and cosine values ​​of each sub-block through a distributed LSTM model pool, and finally fuses the attitude angle estimation values ​​of each sub-block through the dynamic weight of the self-attention mechanism to obtain the final infrared focal plane array attitude estimation result. The method of the present invention can effectively improve the attitude estimation accuracy of the rotating carrier. In addition, during the attitude estimation process, the present invention uses sine and cosine coding to encode the angle value, which can avoid errors caused by angle periodicity. The present invention selects the block mode through a decision network, and then selects a distributed LSTM model according to the block mode, which can effectively utilize the advantages of different block modes and further improve the attitude estimation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 FIG2 is a schematic diagram showing the steps of an infrared focal plane array attitude estimation method provided by an embodiment of the present invention;

[0071] Figure 2 FIG2 is a comparison diagram of pitch angle estimation error curves of the method of the present invention and the extended Kalman filter method in an embodiment of the present invention;

[0072] Figure 3 FIG2 is a comparison diagram of roll angle estimation error curves of the method of the present invention and the extended Kalman filter method in an embodiment of the present invention;

[0073] Figure 4 FIG2 is a time distribution diagram of attitude angle estimation performed by the method of the present invention and the extended Kalman filter method in an embodiment of the present invention;

[0074] Figure 5 FIG2 is a schematic structural diagram of an infrared focal plane array attitude estimation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0076] Example 1

[0077] This embodiment introduces an infrared focal plane array attitude estimation method based on adaptive partitioned distributed LSTM fusion. Starting from the perspective of the time domain signals output by each sensitive element of the sensor, the present invention converts the model-based method into a data-based method. By learning from a large amount of infrared sensor output data, the nonlinear relationship between the infrared focal plane array data and the attitude angle is directly established. Without the need for a precise physical model, the shortcomings of traditional methods such as model mismatch, noise sensitivity, and feature extraction failure are effectively avoided, providing new ideas for infrared attitude measurement methods for rotating carriers.

[0078] like Figure 1 As shown, the specific operating steps of the method of the present invention are as follows:

[0079] Step 1: Use a pre-built infrared attitude measurement system to collect infrared focal plane array data and corresponding carrier attitude angle data, and pre-process the collected infrared focal plane array data. The carrier attitude angle data mainly includes pitch angle and roll angle data.

[0080] Step 1.1. Based on the motion patterns of the rotating vehicle, a hardware-in-the-loop (HIL) device for the infrared attitude measurement system is designed to collect the required data set. The device, consisting of an MLX90640 infrared focal plane array sensor and an STM32H743 microcontroller, is mounted on a three-axis turntable. The turntable is driven to simulate the vehicle's flight attitude, causing the pitch angle to vary within a certain range and the roll angle to continuously change in the positive direction. The infrared focal plane array sensor output and the actual attitude angle are collected as data sets for subsequent training and validation.

[0081] Step 1.2: Use the principal component analysis (PCA) denoising method to remove noise from the collected infrared focal plane array data.

[0082] First, the principal component directions are obtained by calculating the eigenvalue decomposition of the covariance matrix. This projected the data from the high-dimensional space into a low-dimensional subspace, achieving data dimensionality reduction. The original data is then reconstructed using the reduced data and the principal component matrix. At this point, the noise is significantly filtered out because it is distributed among the minor components. The reconstructed data is then added to the previously subtracted mean to restore the original scale.

[0083] Step 2: Extract features from the preprocessed infrared focal plane array data, and use the MLP decision network to select the appropriate block method for adaptive block segmentation based on the extracted features.

[0084] Step 2.1: Use a sliding window method to extract features from each frame or a segment of continuous frames of the infrared focal plane array data to obtain multiple feature vectors. The specific operations are as follows:

[0085] (1) The infrared focal plane array data with a window length of C is formed into a A three-dimensional tensor of , where Indicates the number of rows of the infrared focal plane array sensor, Indicates the number of columns of the infrared focal plane array sensor.

[0086] (2) For the three-dimensional tensor data of each window, extract its statistical features such as overall mean, overall standard deviation, range, frame-by-frame mean sequence, and frame-by-frame standard deviation sequence. The calculation formula is as follows:

[0087] ;

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] in, Indicates the first Frame, Row, No. Column of infrared focal plane array data (i.e., the output value of the infrared focal plane array sensor element), represents the average value of all infrared focal plane array data within the window, represents the standard deviation of all infrared focal plane array data within the window, It represents the difference between the maximum and minimum values ​​of all infrared focal plane array data in the window. Indicates the maximum value of all infrared focal plane array data in the window, Indicates the minimum value of all infrared focal plane array data in the window, Indicates the The average value of all pixel outputs of the frame infrared focal plane array sensor, Indicates the The standard deviation of the output of all bins of the infrared focal plane array sensor.

[0093] (3) Use the statistical features of each window to form a length of The one-dimensional feature vector of , as input for subsequent adaptive block decision.

[0094] .

[0095] Step 2.2: Extract the feature vector from each window , a pre-trained multi-layer perceptron (MLP) decision network is used to achieve adaptive selection of the block method.

[0096] The MLP decision network structure includes an input layer, a hidden layer, and an output layer. The training method of the MLP decision network includes: As the input of the MLP decision network, the infrared focal plane array data of the corresponding time window is used as the input of the pre-trained distributed LSTM model, and different block methods are used to As the output of the MLP decision network, the attitude angle estimation error under different block methods is used as the block effect label to supervise the learning and training process of the MLP decision network. Indicates that each sub-block divided by this block method is OK, Column, please note that, Must be The factors of Must be factors.

[0097] When calling the MLP decision network model, first the extracted feature vector Input the trained MLP decision network, and then use the decision network to select the block method with the highest probability under the current feature input As the optimal block strategy for this window, the formula is as follows:

[0098] ;

[0099] in, Represents the mapping process of the multi-layer perceptron to the input features in the MLP decision network model. Used to normalize the network output into a probability distribution.

[0100] Step 2.3: Optimal block division method according to the current window , divide the current window into several sub-blocks, each sub-block is OK, List.

[0101] Specifically, each frame of infrared focal plane array data after block processing Can be expressed as a The matrix is ​​of the following form:

[0102] ;

[0103] in, Represents the sub-block at the mth row and nth column after block processing, .

[0104] is a The matrix whose elements are extracted from the raw infrared focal plane array data is obtained by the following formula:

[0105] ;

[0106] in, Indicates the first Row, No. Output of a pixel of an infrared focal plane array sensor.

[0107] Through this division, The sub-blocks can completely cover the original Infrared sensor array data.

[0108] Step 3: Based on the infrared focal plane array data of each sub-block after adaptive blocking and the blocking method, call the corresponding model from the distributed long short-term memory network (LSTM) model pool to estimate the sine and cosine values ​​of the attitude angle of each sub-block.

[0109] Step 3.1: Construct the model input based on the LSTM input format using the adaptively segmented infrared focal plane array data of each sub-block, specifically including:

[0110] (1) Flatten each frame of infrared focal plane array data of each sub-block by row. For each sub-block , read the elements line by line starting from the first line, and then arrange these elements into a The one-dimensional vector is then transposed to obtain As the feature vector of one frame of infrared focal plane array data of this sub-block, the formula is as follows:

[0111] ;

[0112] in, Represents a sub-block Middle Rank Column infrared focal plane array data, where the superscript T represents the transpose operation.

[0113] (2) Use the infrared focal plane array data of each sub-block C frame to construct the input sample of the model. Suppose the first sub-block in the window The feature vector of the frame infrared focal plane array data is expressed as , No. The input sample of the frame LSTM model is represented as ,but It is composed of sub-blocks No. Frame and The feature vectors of the C-1 frames before the frame are concatenated. The specific formula is as follows:

[0114] .

[0115] Step 3.2: Call the corresponding model from the distributed LSTM model pool according to different block conditions, and use the selected model to estimate the sine and cosine values ​​of the attitude angle of each sub-block.

[0116] In this invention, a distributed LSTM model pool is pre-built , which contains distributed LSTM models corresponding to different blocking methods. According to the blocking method corresponding to the infrared focal plane array data of the current time window, the corresponding model is called from the model pool to estimate the sine and cosine values ​​of the attitude angle.

[0117] Distributed LSTM model pool It can be expressed as:

[0118] ;

[0119] in, Indicates that the block mode is The corresponding distributed LSTM model.

[0120] Each model in the distributed LSTM model pool They all contain an input layer, two LSTM networks with Dropout regularization, a fully connected layer, and an output layer. The input layer receives the input samples of each sub-block. ,sample Pass through two LSTM networks with Dropout regularization in sequence Then get the hidden state ; followed by a fully connected layer Map the last time step hidden state output by the LSTM layer into features ; Finally, the output layer Output the estimated sine and cosine values ​​of the attitude angle of each sub-block 、 、 and ,in, Represents a sub-block The estimated sine of the pitch angle, Represents a sub-block The estimated value of the pitch angle cosine, Represents a sub-block The estimated value of the sine of the roll angle is Represents a sub-block The estimated value of the cosine of the roll angle is as follows:

[0121] ;

[0122] ;

[0123] .

[0124] During the model training process, the true attitude angle label is re-expressed in the form of sine and cosine encoding using the following formula, and the re-expressed 、 、 and As the true value to supervise model training:

[0125] , .

[0126] Step 4: Based on the fusion function of the self-attention mechanism, the attitude angle sine and cosine values ​​estimated by the distributed LSTM model of each sub-block are fused, and the fused attitude angle sine and cosine values ​​are converted into attitude angles in the angle system to obtain the final infrared focal plane array attitude estimation result.

[0127] Step 4.1: Based on the sine and cosine values ​​of the attitude angles estimated by the distributed LSTM model for each sub-block, obtain the block output matrix corresponding to the window.

[0128] In the above steps, the time series data of the infrared focal plane array in a window has been divided into Sub-blocks, The sub-blocks are estimated by the distributed LSTM network. The attitude angle sine and cosine values ​​of the group are first combined into a vector in the fusion stage. :

[0129] ;

[0130] in, Represents a sub-block A vector of attitude angle sine and cosine estimates.

[0131] Combine the vectors of attitude angle sine and cosine estimates of all sub-blocks into a block output matrix :

[0132] ;

[0133] in, ,and Each row corresponds to a sub-block attitude angle sine and cosine estimation vector, and the block output matrix The comprehensive reflection of the independent prediction results of all sub-blocks.

[0134] Step 4.2: Use block output matrix And the self-attention mechanism calculates the similarity matrix between sub-blocks :

[0135] ;

[0136] in, Represents the attitude angle sine and cosine value vector of the sub-block at row m and column n and the attitude angle sine and cosine value vector of the sub-block in row k and column l The similarity between them is the correlation value between the outputs obtained by scaling the dot product, and the superscript T represents the transpose operation.

[0137] Step 4.3: Similarity matrix conduct Normalize and get the attention weight matrix , the calculation formula is as follows:

[0138] ;

[0139] ;

[0140] in, Indicates that in the fusion sub-block The output result is the sub-block The reference degree of information, that is, the proportion of the information in the final fusion result. The larger the value, the higher the proportion of the block output.

[0141] Step 4.4: Using the attention weight matrix Perform weighted summation on the attitude angle sine and cosine estimation values ​​of each sub-block under the same window to obtain the fused attitude angle sine and cosine estimation value , the calculation formula is as follows:

[0142] ;

[0143] in, Represents a sub-block The fused output vector after weighted fusion of all sub-block information under the self-attention mechanism reflects the output of the sub-block after adaptive fusion of all sub-block information. The superscript T represents the transpose operation.

[0144] Step 4.5: further reduce the weighted results of all sub-blocks to obtain the global fusion output :

[0145] .

[0146] Global fusion output Performing bi-norm normalization to ensure the consistency and numerical stability of the output direction, we can obtain:

[0147] ;

[0148] in, is the normalized fusion vector, 、 、 and They represent the estimated value of the pitch angle sine component, the estimated value of the pitch angle cosine component, the estimated value of the roll angle sine component, and the estimated value of the roll angle cosine component after normalization and fusion, respectively.

[0149] Step 4.6: Normalize the fusion vector Reverse encoding Estimated attitude angle within the range and ,by For example, the reverse encoding method is as follows:

[0150] .

[0151] At this point, the final infrared focal plane array attitude estimation result is output.

[0152] In order to verify the estimation effect of the method of the present invention, the embodiment of the present invention conducted the following experiments:

[0153] In the experiment, the infrared attitude measurement system collected infrared focal plane array data and the corresponding pitch and roll angle data. The sampling frame rate of the infrared focal plane array data and the true attitude data in the collected data set was 64Hz. Each frame of data contained the data collected in real time by the flattened infrared focal plane array sensor and the corresponding true data of the pitch and roll angles. The data set was approximately 37.5 seconds long and contained a total of 2400 data items. The pitch angle range was controlled to continuously change between 30 and 50 degrees, and the roll angle continuously changed between 0 and 360 degrees. The roll angle rate in each cycle changed randomly under reasonable constraints.

[0154] Assume that the window length C=10, use the sliding window to divide the 10 consecutive frames of infrared focal plane array data into a window, and obtain multiple window data; Assume A=24, B=32, and form each window data into a The statistical features of each three-dimensional tensor are calculated using the following formula.

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] ;

[0160] .

[0161] According to the feature vector F, the trained MLP decision network is used to select the block mode to divide the infrared focal plane array data in the window into multiple sub-blocks. In the experiment of the present invention, the block mode includes 、 、 and .

[0162] Assume that the optimal block partitioning method for a window is , according to this block method, the current window is divided into 24 sub-blocks, each sub-block can be expressed as ,in , ; Each sub-block is a The matrix of .

[0163] Each frame of infrared focal plane array data of each sub-block is flattened into a one-dimensional vector of length 32 and then transposed to obtain ,10 frames of infrared focal plane array data from each sub-block are used to construct the model input.,The 24 sub-blocks are estimated by the distributed LSTM model to obtain 24 sets of attitude angle,sine and cosine values.

[0164] Based on the self-attention mechanism, the attitude angle sine and cosine values ​​of the 24 sub-blocks are fused, and the fused attitude angle sine and cosine values ​​are converted into attitude angles in the angle system to obtain the attitude estimation result of the infrared focal plane array.

[0165] In the experiment, the extended Kalman filter (EKF) method was also used to estimate the attitude angle of the same infrared focal plane array data. The estimation results of the carrier pitch angle and roll angle of the method of the present invention and the EKF method are as follows: Figure 2 、 Figure 3 As shown in the figure, the estimation errors are within ±0.8° and ±1.3° respectively. Compared with the EKF method, the average pitch angle error of the method of the present invention is reduced by about 49.7%, and the average roll angle error is reduced by about 21.89%. The attitude angle estimation accuracy of the method of the present invention is much higher than that of the existing technology.

[0166] also, Figure 4 This is the time distribution diagram of attitude angle estimation by the method of the present invention and the extended Kalman filter method, from Figure 4 It can be seen that the average single-frame estimation time of the method of the present invention is about 0.009 seconds, which is about 25% shorter than that of the EKF method. This shows that the method of the present invention effectively improves the estimation accuracy and real-time performance of the attitude angle of the infrared attitude test method.

[0167] Example 2

[0168] Based on the same inventive concept as Example 1, this embodiment introduces an infrared focal plane array attitude estimation device, such as Figure 5 As shown, including:

[0169] The feature extraction module is used to extract features of infrared focal plane array data using a sliding window to obtain feature vectors of multiple windows.

[0170] The blocking module is used to select a blocking method based on the feature vectors of multiple windows and divide the infrared focal plane array data in the window into multiple sub-blocks using the MLP decision network.

[0171] The attitude angle estimation module is used to call the corresponding distributed LSTM model from the distributed LSTM model pool based on the infrared focal plane array data and block division method of each sub-block to estimate the attitude angle sine and cosine values ​​of each sub-block.

[0172] The attitude angle fusion conversion module is used to fuse the attitude angle sine and cosine values ​​of each sub-block based on the self-attention mechanism, and convert the fused attitude angle sine and cosine values ​​into attitude angles in the angle system to obtain the infrared focal plane array attitude estimation result.

[0173] The specific functional implementation of each of the above modules can be found in the relevant content of the method in Example 1 and will not be elaborated on here.

[0174] Example 3

[0175] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the infrared focal plane array attitude estimation method introduced in Example 1 are implemented.

[0176] In summary, the present invention, based on the time-domain signals output by each sensor's sensitive bins, designs a method for attitude estimation of infrared focal plane arrays (IFPAs) based on the fusion of an MLP decision network, a distributed LSTM model pool, and a dynamic weighted self-attention mechanism. By integrating adaptive binning with the self-attention mechanism, this method improves accuracy while effectively leveraging the advantages of different binning methods, providing new insights and methods for attitude measurement of IFPAs.

[0177] In order to address the problem that when using angle values ​​for supervised learning during model training, the roll angle exhibits periodic changes, resulting in deviation calculation errors and poor model training, the present invention uses sine and cosine coding to encode the angle values, avoiding the problem of excessive errors caused by angle periodicity.

[0178] The present invention converts the model-based method into a data-based method, avoids the problem of low estimation accuracy of the EKF method due to inaccurate model, and effectively improves the real-time estimation of the attitude angle of the infrared attitude test method.

[0179] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0180] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0181] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0182] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0183] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A method for estimating an infrared focal plane array attitude, characterized in that: include: The features of infrared focal plane array data are extracted using sliding windows to obtain feature vectors of multiple windows; According to the feature vector of each window, the MLP decision network is used to select the block method corresponding to the window, and the infrared focal plane array data in the window is divided into multiple sub-blocks; According to the infrared focal plane array data and block division method of each sub-block, the corresponding distributed LSTM model is called from the distributed LSTM model pool to estimate the sine and cosine values ​​of the attitude angle of each sub-block; The attitude angle sine and cosine estimation values ​​of each sub-block are fused based on the self-attention mechanism, and the fused attitude angle sine and cosine estimation values ​​are converted into attitude angle estimation values ​​in the angle system to obtain the attitude estimation result of the infrared focal plane array; The attitude angle sine and cosine estimation values ​​of each sub-block are fused based on the self-attention mechanism, and the fused attitude angle sine and cosine estimation values ​​are converted into attitude angle estimation values ​​in an angle system to obtain an infrared focal plane array attitude estimation result, including: Generate the block output matrix corresponding to the window according to the attitude angle sine and cosine estimation values ​​of each sub-block : ； in, Represents the sub-block at row m and column n The estimated sine and cosine values ​​of the attitude angle are vectors, Represents a sub-block The estimated sine of the pitch angle, Represents a sub-block The estimated value of the pitch angle cosine, Represents a sub-block The estimated value of the sine of the roll angle is Represents a sub-block The estimated value of the roll angle cosine is , , M and N are the total number of rows and columns of sub-blocks in the window respectively; Using block output matrix And the self-attention mechanism calculates the similarity matrix between each sub-block in the window : ； in, Represents the attitude angle sine and cosine value vector of the sub-block at row m and column n and the attitude angle sine and cosine value vector of the sub-block in row k and column l The similarity between them, T represents the transposition operation; The similarity matrix between each sub-block in the window conduct Normalize and get the attention weight matrix of each sub-block in the window : ； ； in, Indicates the degree of reference to the sub-block information of the kth row and lth column when fusing the output results of the sub-block of the mth row and nth column; Using the attention weight matrix Perform weighted summation on the attitude angle sine and cosine estimation values ​​of each sub-block in the window to obtain the fused attitude angle sine and cosine estimation value : ； in, It represents the fusion output vector of the sub-block of the mth row and nth column after weighted fusion of all sub-block information under the self-attention mechanism, and T represents the transpose operation; According to the fused attitude angle sine and cosine values Perform reduction processing to obtain global fusion output : ； Global fusion output Perform two-norm normalization: ； in, represents the normalized fusion vector, 、 、 and They represent the estimated value of the pitch angle sine component, the estimated value of the pitch angle cosine component, the estimated value of the roll angle sine component, and the estimated value of the roll angle cosine component after normalization and fusion respectively; Normalize the fusion vector Reverse encoding The attitude angle estimation value within the range is used as the attitude estimation result of the infrared focal plane array.

2. The infrared focal plane array attitude estimation method according to claim 1, characterized in that The method of extracting features of infrared focal plane array data using a sliding window to obtain feature vectors of multiple windows includes: Using a sliding window to divide the infrared focal plane array data into multiple window data; Each window data is composed of a A three-dimensional tensor of , where Indicates the number of rows of the infrared focal plane array sensor, represents the number of columns of the infrared focal plane array sensor, and C represents the window length; Calculating statistical features of the three-dimensional tensor data of each window, wherein the statistical features include overall mean, overall standard deviation, range, frame-by-frame mean sequence, and frame-by-frame standard deviation sequence; Concatenate the statistical features of each window to form a one-dimensional feature vector of the window data .

3. The infrared focal plane array attitude estimation method according to claim 1, wherein A sliding window is used to extract the features of historical infrared focal plane array data to obtain historical feature vectors of multiple windows. The historical feature vectors of different windows are used as the input of the MLP decision network, and the probabilities of different blocking methods are used as the output of the MLP decision network. The attitude angle estimation errors under different blocking methods are used as the blocking effect labels to train the MLP decision network and obtain a trained MLP decision network.

4. The infrared focal plane array attitude estimation method according to claim 3, wherein: For each window’s historical feature vector, the partitioning method with the highest probability output by the MLP decision network is selected as the optimal partitioning method for the window. Dividing the infrared focal plane array data in the window into several sub-blocks according to the optimal block division method; Assume that the optimal block partitioning method is ,in, 、 Respectively represent the number of rows and columns of each sub-block under the optimal blocking method, and each frame of infrared focal plane array data in the window is expressed as follows after blocking processing: ； in, represents the matrix after block processing, Represents the sub-block at the mth row and nth column after block processing, , , M and N are the total number of rows and columns of sub-blocks in the window respectively, , A and B represent the number of rows and columns of the infrared focal plane array sensor, respectively; is a The matrix expression is as follows: ； in, Indicates the first Row, No. The value of the column, , .

5. The infrared focal plane array attitude estimation method according to claim 1, wherein The method of estimating the attitude angle sine and cosine values ​​of each sub-block by calling the corresponding distributed LSTM model from the distributed LSTM model pool according to the infrared focal plane array data and the block division method of each sub-block includes: Select the distributed LSTM model corresponding to the block mode from the distributed LSTM model pool; Convert the infrared focal plane array data of each sub-block into model input data according to the input format of the distributed LSTM model; The selected distributed LSTM model is used to process the model input data to obtain the attitude angle sine and cosine estimation values ​​of each sub-block.

6. The infrared focal plane array attitude estimation method according to claim 5, characterized in that The infrared focal plane array data of each sub-block is converted into model input data according to the input format of the distributed LSTM model, including: Flatten each frame of infrared focal plane array data of each sub-block by row to obtain the feature vector of each frame: ； in, Represents the sub-block at row m and column n A frame feature vector of Represents a sub-block Middle Row, No. Columns of infrared focal plane array data, , , M and N are the total number of rows and columns of sub-blocks in the window, and T represents the transpose operation; use Moment sub-block The feature vectors of the current frame and the historical C-1 frame are concatenated. The input data of the time-distributed LSTM model is expressed as follows: ； in, express The input data of the time-distributed LSTM model, express Moment sub-block The feature vector of the current frame in , C represents the window length.

7. The infrared focal plane array attitude estimation method according to claim 5, characterized in that: Each distributed LSTM model in the distributed LSTM model pool includes an input layer, two layers of LSTM networks with Dropout regularization, a fully connected layer, and an output layer; After the model input data enters the distributed LSTM model through the input layer, it passes through two LSTM networks with Dropout regularization in sequence. Then get the hidden state; fully connected layer Mapping hidden states to features; output layer Output the sine and cosine values ​​of the attitude angle of each sub-block according to the mapping features; the formula is as follows: ； ； ； in, express The sub-block at the mth row and nth column at the moment The corresponding model input data, express The corresponding hidden state, express The corresponding mapping features, Represents a sub-block The estimated sine of the pitch angle, Represents a sub-block The estimated value of the pitch angle cosine, Represents a sub-block The estimated value of the sine of the roll angle is Represents a sub-block The estimated value of the roll angle cosine is , , M and N are the total number of rows and columns of sub-blocks in the window respectively.

8. An infrared focal plane array attitude estimation device based on the infrared focal plane array attitude estimation method according to claim 1, characterized in that: include: A feature extraction module is used to extract features of infrared focal plane array data using a sliding window to obtain feature vectors of multiple windows; A block division module is used to select a block division method corresponding to a window by using an MLP decision network according to the feature vectors of each window, and divide the infrared focal plane array data in the window into multiple sub-blocks; The attitude angle estimation module is used to call the corresponding distributed LSTM model from the distributed LSTM model pool based on the infrared focal plane array data and block division method of each sub-block to estimate the attitude angle sine and cosine values ​​of each sub-block; The attitude angle fusion conversion module is used to fuse the attitude angle sine and cosine estimation values ​​of each sub-block based on the self-attention mechanism, and convert the fused attitude angle sine and cosine estimation values ​​into attitude angle estimation values ​​in the angle system to obtain the infrared focal plane array attitude estimation result.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the infrared focal plane array attitude estimation method described in any one of claims 1 to 7 are implemented.

Images