A method and device for estimating a projectile attitude angle based on a space-time fusion double-branch network

Abstract

The application discloses a method and device for estimating the attitude angle of a projectile based on a space-time fusion double-branch network, and belongs to the technical field of carrier attitude estimation. The method comprises the following steps: using a pitch angle branch of the space-time fusion double-branch network to perform feature coding and time sequence modeling on an infrared grayscale image sequence to obtain time sequence enhanced pitch angle features; using a roll angle branch of the space-time fusion double-branch network to perform feature coding and time sequence modeling on the infrared grayscale image sequence to obtain time sequence enhanced roll angle features; inputting the time sequence enhanced pitch angle features and the time sequence enhanced roll angle features into corresponding multi-layer perceptrons respectively to obtain space-time fused pitch angle cosine and sine representation results and roll angle cosine and sine representation results; and using four-quadrant inverse tangent functions to solve the pitch angle cosine and sine representation results and the roll angle cosine and sine representation results respectively to obtain an estimated value of the pitch angle of the projectile and an estimated value of the roll angle of the projectile. The application can realize real-time and high-precision attitude estimation of a rotating projectile.

Description

Technical Field

[0001] This invention relates to a method and apparatus for estimating the attitude angle of a projectile based on a spatiotemporal fusion dual-branch network, belonging to the field of carrier attitude estimation technology. Background Technology

[0002] In modern aerospace, precise attitude control of rotating projectiles is crucial. Traditional inertial sensors are easily affected by complex environmental factors when measuring the attitude of rotating projectiles, resulting in large measurement errors and failing to meet the requirements of high-precision attitude control. Infrared focal plane arrays, as an important infrared detection technology, have shown great application potential in the attitude measurement of rotating projectiles due to their advantages such as high sensitivity, high resolution, and ability to acquire target information in complex environments.

[0003] Numerous studies have been conducted on infrared-based projectile attitude measurement methods. For instance, Chinese invention patent CN120579150A discloses an infrared focal plane array attitude estimation method and device, which uses adaptive block partitioning, a distributed LSTM model, and a self-attention mechanism dynamic weight fusion technique to calculate the carrier's attitude. Another Chinese invention patent CN114413908A discloses a four-axis long-wave infrared radiation attitude compensation method under solar interference, addressing the issues of low attitude measurement accuracy and weak anti-interference capability for rotating aircraft under solar infrared radiation interference. However, these infrared focal plane array-based projectile attitude estimation methods largely rely on the time-domain signal of the infrared focal plane array, focusing only on single-dimensional information extraction. They fail to fully utilize the spatial structure and frequency-domain energy characteristics inherent in the infrared focal plane array image, making it difficult to meet the real-time, high-precision attitude estimation requirements of high-speed rotating projectiles. Summary of the Invention

[0004] The purpose of this invention is to provide a projectile attitude angle estimation method and device based on a spatiotemporal fusion dual-branch network. By fully exploiting the spatial structure and frequency domain energy features of a single frame image through the spatiotemporal fusion dual-branch network, a real-time, high-precision attitude estimation of a rotating projectile can be achieved.

[0005] To achieve the above objectives / to solve the above technical problems, the present invention is implemented using the following technical solution:

[0006] In a first aspect, the present invention provides a projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network, comprising:

[0007] The infrared grayscale image sequence is obtained by splitting and linearly transforming the time-domain signal of the infrared focal plane array.

[0008] The pitch angle related spatial features of infrared grayscale image sequences are extracted by using the pitch angle branch of the spatiotemporal fusion dual-branch network, and temporal modeling is performed to obtain temporally enhanced pitch angle features;

[0009] The roll angle related spatial features of infrared grayscale image sequences are extracted by utilizing the roll angle branch of the spatiotemporal fusion dual-branch network, and temporal modeling is performed to obtain temporally enhanced roll angle features.

[0010] The temporal enhanced pitch angle feature and the temporal enhanced roll angle feature are respectively input into the corresponding multilayer perceptron to obtain the spatiotemporal fusion pitch angle sine and cosine representation results and roll angle sine and cosine representation results;

[0011] By using the four-quadrant arctangent function to solve the sine and cosine representations of the pitch angle and the roll angle, respectively, the estimated values ​​of the projectile's pitch angle and roll angle are obtained.

[0012] Furthermore, the process of extracting pitch angle-related spatial features from the infrared grayscale image sequence using the pitch angle branch of the spatiotemporal fusion dual-branch network and performing temporal modeling to obtain temporally enhanced pitch angle features includes:

[0013] Multi-scale frequency domain amplitude modeling is performed on a single frame of infrared grayscale image in an infrared grayscale image sequence to obtain the amplitude spectrum at each scale;

[0014] The amplitude spectra at each scale are input into a convolutional neural network, and high-dimensional features in the frequency domain are obtained through feature extraction.

[0015] Row vector statistics are performed on a single frame of infrared grayscale image, and then the row vector statistical features are obtained by mapping using a multilayer perceptron.

[0016] By splicing together high-dimensional features in the frequency domain and statistical features of row vectors, the spatial features related to the pitch angle of a single frame are obtained.

[0017] Based on the spatial features related to the pitch angle of each frame in the infrared grayscale image sequence, a temporal feature sequence of pitch angle is constructed.

[0018] The pitch angle temporal feature sequence is input into a pitch angle temporal convolution module composed of stacked multi-layer one-dimensional dilated convolutional residual blocks for temporal modeling, resulting in a pitch angle temporal enhanced feature sequence:

[0019] ;

[0020] in, After representing the timing model Temporal enhanced pitch angle feature sequence at time step, Indicates the first time series modeling Temporal enhancement features of pitch angle in frame images, The length of the infrared grayscale image sequence;

[0021] Extract the last time step feature of the pitch angle temporal enhancement feature sequence as the temporal enhancement pitch angle feature. :

[0022] .

[0023] Furthermore, the step of performing multi-scale frequency domain amplitude modeling on a single frame of infrared grayscale image in the infrared grayscale image sequence to obtain the amplitude spectrum at each scale includes:

[0024] Using a two-dimensional bilinear downsampling operator to convert a single frame of infrared grayscale image Constructing a multi-scale image:

[0025] ;

[0026] ;

[0027] in, This is the image after downsampling when the scale factor is s. , This represents a two-dimensional bilinear downsampling operator with a scaling factor of s. This represents the image after downsampling when the scale factor is s. mid-coordinate point The pixel grayscale value at that location, and These represent the height and width of the image, respectively. Image before downsampling mid-coordinate point The pixel grayscale value at that location, The weights of the two-dimensional bilinear downsampling operator;

[0028] Weight The calculation formula is:

[0029] ;

[0030] in, This is for calculating the maximum value.

[0031] Perform two-dimensional fast Fourier transform on images at each scale:

[0032] ;

[0033] in, This represents the coordinates of the image with scale factor s after undergoing a two-dimensional fast Fourier transform. Frequency domain coefficients at that location, For frequency domain coordinates, Represents the coordinates of points in an image with a scale factor of s. The pixel grayscale value at that location, The imaginary unit;

[0034] Define the amplitude spectrum of images at each scale:

[0035] ;

[0036] in, The Fourier amplitude spectrum of the image with scale factor s. The result of the two-dimensional Fast Fourier Transform of the image with a scale factor of s. , For complex number modulus operations;

[0037] A nonlinear transformation of the amplitude spectrum is performed using a logarithmic mapping:

[0038] ;

[0039] in, The amplitude spectrum after nonlinear transformation. It is a logarithmic function.

[0040] Furthermore, the step of inputting the amplitude spectrum at each scale into a convolutional neural network and obtaining high-dimensional frequency domain features through feature extraction includes:

[0041] Amplitude spectra at various scales Input convolutional neural network Extract frequency-space joint features at each scale. ;

[0042] Frequency-spatial joint features of the original scale image Weighted processing is performed to obtain The formula is as follows:

[0043] ;

[0044] ;

[0045] in, The original scale frequency domain features are weighted. For element-wise multiplication, This is the confidence plot corresponding to the original scale. This represents the Sigmoid activation function. This is a 1×1 convolution operation;

[0046] Global average pooling is performed on features at each scale to obtain feature vectors at each scale. The formula is as follows:

[0047] ;

[0048] in, The feature vector corresponding to the scale factor s, The original scale frequency domain features after weighting mid-coordinate point eigenvalues ​​at that location scale factor Corresponding frequency-space joint features mid-coordinate point eigenvalues ​​at that location and These represent the height and width of the image, respectively.

[0049] By concatenating feature vectors at various scales and fusing them through a fully connected layer, high-dimensional features in the frequency domain can be obtained. :

[0050] ;

[0051] in, It is a mapping module consisting of fully connected layers and nonlinear activation functions.

[0052] Furthermore, the step of performing row vector statistics on a single frame of infrared grayscale image and then using a multilayer perceptron to map and obtain row vector statistical features includes:

[0053] Calculate a single frame of infrared grayscale image row mean vector :

[0054] ;

[0055] ;

[0056] in, Represents the first in an infrared grayscale image The mean of all elements in a row. Represents the first in an infrared grayscale image line, number The pixel values ​​of the column;

[0057] Using a multilayer perceptron to analyze the row mean vector Mapping yields row vector statistical features .

[0058] Furthermore, the method of extracting roll angle-related spatial features of infrared grayscale image sequences using the roll angle branch of the spatiotemporal fusion dual-branch network and performing temporal modeling to obtain temporally enhanced roll angle features includes:

[0059] For a single frame of infrared grayscale image Intra-sample standardization is performed to obtain the standardized image. :

[0060] ;

[0061] in, To prevent extremely small constants with a denominator of zero, , Infrared grayscale images The mean and standard deviation;

[0062] Standardized image Inputting into a column-oriented one-dimensional convolutional neural network, the column-oriented convolutional features are obtained through feature extraction. :

[0063] ;

[0064] in, The GELU nonlinear activation function is used. This represents a column-oriented one-dimensional convolution operator. and These represent the column-oriented one-dimensional convolution weights and biases, respectively.

[0065] Column-wise convolution features One-dimensional global average pooling is used to obtain the column structure vector. :

[0066] ;

[0067] in, This represents a one-dimensional global average pooling operation. Represents column-wise convolutional features The Middle Feature values ​​at each position, Indicates the height of the image;

[0068] Standardized image Transpose to get:

[0069] ;

[0070] in, This represents the transposed image;

[0071] Transposed image Inputting into a row-oriented one-dimensional convolutional neural network, the row-oriented convolutional features are obtained through feature extraction. :

[0072] ;

[0073] in, This represents a row-oriented one-dimensional convolution operator. and These represent the row-directed one-dimensional convolution weights and biases, respectively.

[0074] row-wise convolutional features One-dimensional global average pooling is used to obtain the row structure vector. :

[0075] ;

[0076] in, Representing row-wise convolutional features The Middle Feature values ​​at each position, Indicates the width of the image;

[0077] Concatenate column structure vectors with row structure vector The spatial features related to the roll angle of a single frame are obtained.

[0078] Based on the spatial features related to the roll angle of each frame in the infrared grayscale image sequence, a roll angle temporal feature sequence is constructed.

[0079] The roll angle temporal feature sequence is input into a roll angle temporal convolutional module composed of stacked multi-layer one-dimensional dilated convolutional residual blocks for temporal modeling, resulting in a roll angle temporal enhanced feature sequence:

[0080] ;

[0081] in, After representing the timing model Temporal enhanced roll angle feature sequence at time step, Indicates the first time series modeling Temporal enhancement features of roll angle in frame images, The length of the infrared grayscale image sequence;

[0082] Extract the last time step feature of the roll angle temporal enhancement feature sequence as the temporal enhancement roll angle feature. :

[0083] .

[0084] Furthermore, the pitch angle features will be enhanced temporally. Temporally enhanced roll angle features Input the corresponding multilayer perceptrons respectively , The sine and cosine representations of the attitude angles after spatiotemporal fusion are obtained as follows:

[0085] ;

[0086] in, The result is represented by the sine and cosine of the pitch angle at time t after spatiotemporal fusion. The result is represented by the sine and cosine of the roll angle at time t after spatiotemporal fusion.

[0087] Furthermore, the step of using the four-quadrant arctangent function to solve the sine and cosine representations of the pitch angle and roll angle, respectively, to obtain the estimated values ​​of the projectile's pitch angle and roll angle, includes:

[0088] Based on the sine and cosine representations of the pitch and roll angles, the calculated pitch and roll angle values ​​are obtained:

[0089] ;

[0090] in, and These are the calculated pitch angle and roll angle values ​​at time t, respectively. The arctangent function is in the fourth quadrant, and its output quadrant is... , and Representing the pitch angle, sine and cosine respectively, and representing the result. The sine and cosine components, and The results represent the roll angle (sine and cosine). The sine and cosine components;

[0091] Pitch angle calculation value and roll angle calculation value Angle range conversion The estimated pitch angle of the projectile is obtained. and roll angle estimate :

[0092] ;

[0093] .

[0094] Secondly, the present invention provides a projectile attitude angle estimation device based on a spatiotemporal fusion dual-branch network, comprising:

[0095] The preprocessing module is used to split and linearly transform the time-domain signal of the infrared focal plane array to obtain an infrared grayscale image sequence;

[0096] The pitch angle branch module is used to extract pitch angle-related spatial features of infrared grayscale image sequences using the pitch angle branch of the spatiotemporal fusion dual-branch network, and perform temporal modeling to obtain temporally enhanced pitch angle features;

[0097] The roll angle branch module is used to extract roll angle-related spatial features of infrared grayscale image sequences using the roll angle branch of the spatiotemporal fusion dual-branch network, and perform temporal modeling to obtain temporally enhanced roll angle features.

[0098] The attitude angle calculation module is used to input the temporal enhanced pitch angle features and temporal enhanced roll angle features into the corresponding multilayer perceptrons to obtain the spatiotemporally fused pitch angle sine and cosine representations and roll angle sine and cosine representations. The four-quadrant arctangent function is used to calculate the pitch angle sine and cosine representations and roll angle sine and cosine representations respectively to obtain the estimated pitch angle and roll angle of the projectile.

[0099] Thirdly, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network provided in the first aspect.

[0100] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0101] This invention proposes a projectile attitude angle estimation method and apparatus based on a spatiotemporal fusion dual-branch network. It reconstructs the temporal domain signal of an infrared focal plane array into a grayscale image, fully exploits the spatial structure and frequency domain energy features of a single-frame image through a spatiotemporal fusion dual-branch network, and combines this with a temporal convolutional network to mine cross-frame temporal correlations. This multi-dimensional information mining significantly improves the accuracy and real-time performance of projectile attitude estimation in dynamic scenes. Furthermore, considering the differentiated representation patterns of pitch and roll angles, this invention designs independent branch extraction paths to effectively avoid feature confusion and ensure the relevance and effectiveness of feature representation. Attached Figure Description

[0102] Figure 1 The diagram shows the steps of a projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network provided by the present invention.

[0103] Figure 2 The diagram shown is a flowchart of the projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network in an embodiment of the present invention.

[0104] Figure 3 The diagram shown is a schematic representation of the projectile attitude angle in an embodiment of the present invention;

[0105] Figure 4 The figure shown is a comparative diagram of the error curves of the method of the present invention and the distributed LSTM method for calculating the projectile pitch angle in an embodiment of the present invention.

[0106] Figure 5 The figure shown is a comparative diagram of the error curves of the method of the present invention and the distributed LSTM method for calculating the roll angle of the projectile in an embodiment of the present invention.

[0107] Figure 6 The diagram shows a comparison of the estimation time of a single frame image when the method of the present invention and the distributed LSTM method are used to calculate the attitude angle of the projectile in an embodiment of the present invention.

[0108] Figure 7 The diagram shows a schematic of a projectile attitude angle estimation device based on a spatiotemporal fusion dual-branch network provided by the present invention. Detailed Implementation

[0109] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0110] Example 1

[0111] This embodiment introduces a projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network. The method of this invention constructs a spatiotemporal fusion dual-branch network (TSF-DBNet) that integrates temporal and spatial frequency domains to achieve collaborative modeling of spatial and temporal dynamic features of multiple frames of infrared images. The dual-branch network is used to estimate the pitch and roll angles of the projectile, making full use of the multi-dimensional information of the infrared focal plane array signal to complete the high-precision estimation of the projectile attitude angle.

[0112] like Figure 1 , Figure 2 As shown, the method of the present invention specifically includes the following steps:

[0113] Step 1: Decompose and linearly transform the time-domain signal of the infrared focal plane array to obtain an infrared grayscale image sequence.

[0114] Step 2: Utilize the pitch angle branch of TSF-DBNet to perform feature encoding and temporal modeling on the infrared grayscale image, obtaining temporally enhanced pitch angle features. Specifically, the length of... The infrared grayscale image sequence is input into the pitch angle branch of TSF-DBNet, and the pitch angle spatial frequency domain feature extraction branch with shared parameters is used (i.e. Figure 2 The multi-scale frequency domain replication, convolutional neural network, row vector statistics and multilayer perceptron in the sequence are used to encode the images in the sequence frame by frame to obtain the pitch angle related spatial features of each frame, and organize them into a pitch angle temporal feature sequence in chronological order; the pitch angle temporal feature sequence is input into the pitch angle temporal convolution module for temporal modeling, mining cross-frame temporal correlation information, and extracting the features of the last time step to obtain the temporally enhanced pitch angle features.

[0115] Step 3: Utilize the roll angle branch of TSF-DBNet to perform feature encoding and temporal modeling on the infrared grayscale image, obtaining temporally enhanced roll angle features. Specifically, the length of... The infrared grayscale image sequence is input into the roll angle branch of TSF-DBNet, and the roll angle spatial feature extraction branch with shared parameters (i.e. Figure 2 The row-oriented one-dimensional convolutional neural network and the column-oriented one-dimensional convolutional neural network in the sequence encode the images in the sequence frame by frame to obtain the roll angle related spatial features of each frame, and organize them into a roll angle temporal feature sequence in chronological order; the roll angle temporal feature sequence is input into the roll angle temporal convolution module for temporal modeling, mining cross-frame temporal correlation information, and extracting the features of the last time step to obtain temporally enhanced roll angle features.

[0116] Step 4: Input the temporal enhanced pitch angle feature and the temporal enhanced roll angle feature into the corresponding multilayer perceptron to obtain the spatiotemporal fusion pitch angle sine and cosine representation results and roll angle sine and cosine representation results.

[0117] Step 5: Use the four-quadrant arctangent function to solve the sine and cosine representations of the pitch angle and roll angle, respectively, to obtain the final estimated values ​​of the projectile's pitch and roll angles.

[0118] In this embodiment of the invention, step 2 is specifically performed as follows:

[0119] Step 2.1: For a single frame of infrared grayscale image First, multi-scale frequency domain amplitude modeling is performed on it.

[0120] Scale factor is constructed using a two-dimensional bilinear downsampling operator. Multiscale images , The scale factor is represented as Two-dimensional bilinear downsampling operator.

[0121] The calculation formula for the two-dimensional bilinear downsampling operator is as follows:

[0122] ;

[0123] in, This represents the image after downsampling when the scale factor is s. mid-coordinate point The pixel grayscale value at that location, and These represent the height and width of the image, respectively. Image before downsampling mid-coordinate point The pixel grayscale value at that location, The weights are those of the two-dimensional bilinear downsampling operator.

[0124] Weight Determined by the bilinear kernel function, the formula is as follows:

[0125] ;

[0126] Then, a two-dimensional fast Fourier transform is performed on the images at each scale to realize the conversion of the image from the spatial domain to the frequency domain, so that the difference in global energy distribution corresponding to the pitch angle change is more prominent in the low-frequency and mid-low-frequency components.

[0127] The formula for the two-dimensional Fast Fourier Transform is as follows:

[0128] ;

[0129] in, The scale factor is represented as The coordinates of the image after two-dimensional fast Fourier transform Frequency domain coefficients at that location, For frequency domain coordinates, The scale factor is represented as coordinates of the image The pixel grayscale value at that location, It is the imaginary unit.

[0130] The amplitude spectrum of the image at each scale is defined based on the frequency domain coefficients:

[0131] ;

[0132] in, The scaling factor is The Fourier amplitude spectrum of the image. The scaling factor is The results of the two-dimensional fast Fourier transform of the image. , This is for modulo operations on complex numbers.

[0133] A nonlinear transformation of the amplitude spectrum using a logarithmic map is employed to compress the dynamic range of the amplitude, resulting in:

[0134] ;

[0135] in, The amplitude spectrum after nonlinear transformation. It is a logarithmic function.

[0136] Step 2.2: Process the amplitude spectrum at each scale. Input convolutional neural network Extracting joint frequency-space features The formula is as follows:

[0137] ;

[0138] in, scale factor The corresponding frequency-space joint features, This refers to the operation of a convolutional neural network, which specifically consists of multiple layers of two-dimensional convolution, nonlinear activation functions, and normalization operations.

[0139] Since different frequency components contribute differently to the pitch angle calculation, the method of this invention utilizes the original scale frequency domain features with the highest spatial resolution and the most complete frequency domain distribution information. A spatial confidence weighting mechanism is introduced above, through Convolution generates a confidence map, using the following formula:

[0140] ;

[0141] in, This is the confidence plot corresponding to the original scale. This represents the Sigmoid activation function. This is a 1×1 convolution operation.

[0142] Will As a region weight for the original scale frequency domain features By performing element-wise weighted summation, we obtain:

[0143] ;

[0144] in, The original scale frequency domain features are weighted. This is element-wise multiplication.

[0145] Global average pooling is performed on features at each scale to obtain feature vectors at each scale. The formula is as follows:

[0146] ;

[0147] in, The feature vector corresponding to the scale factor s, The original scale frequency domain features after weighting mid-coordinate point eigenvalues ​​at that location scale factor Corresponding frequency-space joint features mid-coordinate point The eigenvalue at that location.

[0148] By concatenating feature vectors at various scales and fusing them through a fully connected layer, high-dimensional features in the frequency domain can be obtained. :

[0149] ;

[0150] in, It is a mapping module consisting of fully connected layers and nonlinear activation functions.

[0151] Step 2.3: For a single frame of infrared grayscale image Perform row vector statistics.

[0152] Calculate a single frame of infrared grayscale image row mean vector :

[0153] ;

[0154] in, Indicates the first in the image The mean of all elements in a row.

[0155] ;

[0156] in, Indicates the first in the image line, number The pixel grayscale value of the column.

[0157] Step 2.4: Use a multilayer perceptron (MLP) to process the row mean vector. Mapping yields row vector statistical features .

[0158] Step 2.5, Spatial Feature Assembly: Assembly and The spatial features related to the pitch angle of a single frame are obtained. :

[0159] ;

[0160] in, This indicates a vector concatenation operation.

[0161] Step 2.6: Repeat steps 2.1-2.5 to obtain the pitch angle related spatial features of each frame in the infrared grayscale image sequence, and then construct the pitch angle temporal feature sequence.

[0162] All frames in the sequence are encoded using the same set of pitch angle spatial frequency domain feature extraction branch parameters, and the pitch angle-related spatial features of each frame are organized in chronological order into a pitch angle temporal feature sequence:

[0163] ;

[0164] in, express The pitch angle time series characteristic sequence at time point, The length of the infrared grayscale image sequence. Indicates the first The pitch angle-related spatial feature vector of the frame image. This is a vector transpose operation.

[0165] Step 2.7: Convert the pitch angle time series feature sequence The input consists of a pitch angle temporal convolution module composed of stacked one-dimensional dilated convolution residual blocks. Temporal modeling is performed to mine cross-frame temporal correlation information. This process can be represented as:

[0166] ;

[0167] in, After representing the timing model Temporal enhanced pitch angle feature sequence at time step, This represents the pitch angle temporal convolution operation.

[0168] The formula for one-dimensional dilated convolution in the pitch angle temporal convolution module is:

[0169] ;

[0170] in, Given the input time-series feature sequence, For convolution weights, For output features, The kernel size is [size]. The time step index for the output feature sequence. The expansion rate is denoted as .

[0171] The pitch angle temporal enhancement feature sequence after temporal modeling can be written as:

[0172] ;

[0173] in, Indicates the first time series modeling Temporal enhancement features of the pitch angle of frame images.

[0174] The feature at the last time step of the feature sequence after temporal modeling is extracted as the final temporal-enhanced pitch angle feature. :

[0175] ;

[0176] In this embodiment of the invention, step 3 is specifically performed as follows:

[0177] Step 3.1: Convert the single-frame infrared grayscale image Input a column-oriented one-dimensional convolutional neural network and obtain a column-oriented structure vector through feature extraction.

[0178] For a single frame of infrared grayscale image In-sample standardization is performed using the following formula:

[0179] ;

[0180] in, For the standardized image, To prevent extremely small constants with a denominator of zero, , They are respectively The mean and standard deviation are calculated using the following formulas:

[0181] ;

[0182] ;

[0183] Standardized image As input to a column-oriented one-dimensional convolution, Each row is treated as an input channel, and a one-dimensional convolution is performed along the column direction to obtain the column-directed convolution features. :

[0184] ;

[0185] in, The GELU nonlinear activation function is used. This represents a column-oriented one-dimensional convolution operator. and These represent the column-oriented one-dimensional convolution weights and biases, respectively.

[0186] Then, the column structure vector is obtained through one-dimensional global average pooling. :

[0187] ;

[0188] in, This represents a one-dimensional global average pooling operation. In the column-wise convolution feature U, the first... The feature values ​​at each position.

[0189] Step 3.2: Convert the single-frame infrared grayscale image Input a row-oriented one-dimensional convolutional neural network and obtain row-oriented structure vectors through feature extraction.

[0190] To symmetrically model structural changes along the row direction, the standardized... transpose to get :

[0191] ;

[0192] Will As input to a row-oriented one-dimensional convolution, Each column is treated as an input channel, and a one-dimensional convolution is performed along the row direction to obtain the row-directed convolution features. :

[0193] ;

[0194] in, The GELU nonlinear activation function is used. This represents a row-oriented one-dimensional convolution operator. and These represent the row-directed one-dimensional convolution weights and biases, respectively.

[0195] Then, the row structure vector is obtained through one-dimensional global average pooling. :

[0196] ;

[0197] in, Representing row-wise convolutional features The Middle The feature values ​​at each position.

[0198] Step 3.3, Spatial Feature Concatenation: Concatenating Column-Oriented Structure Vectors with row structure vector The spatial features related to the roll angle of a single frame were obtained. :

[0199] ;

[0200] in, This indicates a vector concatenation operation.

[0201] Step 3.4: Repeat steps 3.1-3.3 to obtain the roll angle related spatial features of each frame in the infrared grayscale image sequence and construct the roll angle temporal feature sequence.

[0202] All frames in the sequence are encoded frame-by-frame using a parameter-shared roll angle spatial feature extraction branch. The roll angle-related spatial features of each frame are then organized in chronological order into a roll angle temporal feature sequence.

[0203] ;

[0204] in, Let be the roll angle time series characteristic sequence at time t. For the first The roll angle-related spatial feature vector of the frame image.

[0205] Step 3.5: Stream the roll angle time-series feature sequence. The input consists of a roll angle temporal convolution module composed of stacked one-dimensional dilated convolution residual blocks. Temporal modeling is performed to mine cross-frame temporal correlation information. This process can be represented as:

[0206] ;

[0207] in, After representing the timing model Temporal enhanced roll angle feature sequence at time step, This indicates a roll angle temporal convolution operation.

[0208] The temporal enhancement features after temporal modeling can be written as:

[0209] ;

[0210] in, Indicates the first time series modeling Temporal enhancement features of roll angle in frame images.

[0211] The features of the last time step of the feature sequence after temporal modeling are extracted and used as the temporal enhanced roll angle features. :

[0212] ;

[0213] In step 4, the time-series enhanced pitch angle features are... Temporally enhanced roll angle features Input the corresponding multilayer perceptrons respectively , The sine and cosine representations of the attitude angles after spatiotemporal fusion are obtained as follows:

[0214] ;

[0215] in, The result is represented by the sine and cosine of the pitch angle at time t after spatiotemporal fusion. The result is represented by the sine and cosine of the roll angle at time t after spatiotemporal fusion.

[0216] In step 5, firstly, based on the sine and cosine representations of the pitch and roll angles, the four-quadrant arctangent function is used to calculate the pitch and roll angle values:

[0217] ;

[0218] in, and These are the calculated pitch angle and roll angle values ​​at time t, respectively. The arctangent function is in the fourth quadrant, and its output quadrant is... , and Representing the pitch angle, sine and cosine respectively, and representing the result. The sine and cosine components, and The results represent the roll angle (sine and cosine). The sine and cosine components.

[0219] Then, calculate the pitch angle. and roll angle calculation value Angle range conversion Add angle After correction, the final estimated pitch angle of the rotating projectile is obtained. and roll angle estimate :

[0220] ;

[0221] ;

[0222] To verify the effectiveness of the method of the present invention, the following comparative experiments are provided in the embodiments of the present invention:

[0223] The attitude angle of the rotating projectile is as follows Figure 3 As shown, where, The reference coordinate system representing the rotating projectile. The projectile coordinate system represents the rotating projectile. Indicates the pitch angle of a rotating projectile. This indicates the roll angle of a rotating projectile.

[0224] This invention employs an attitude measurement device with a single-axis infrared focal plane array sensor array mounted on a three-axis turntable for a semi-physical experiment. A dataset is collected, and the time-domain signal of the infrared focal plane array is split according to sampling time points. The surface element output at each moment is reshaped into an infrared grayscale image sequence through dimensionality conversion. Based on the experimentally calibrated amplitude and deviation, a linear transformation is used to normalize and map the infrared focal plane array data into grayscale images. The sampling frequency of the infrared focal plane array data and the actual attitude data in the collected dataset is 32Hz. Each frame of data contains data collected by the infrared focal plane array sensor and corresponding true labels for pitch and roll angles. The dataset duration is 75 seconds, containing 2400 data points, with the pitch angle continuously varying between 30° and 50°, and the roll angle continuously varying between 0° and 360°.

[0225] Based on infrared grayscale image sequences, the projectile attitude angle is estimated using the method of this invention. At the same time, the method disclosed in Chinese Invention Patent No. CN120579150A (hereinafter referred to as the distributed LSTM method) is also used to estimate the projectile attitude angle, so as to compare the attitude angle estimation effects of different methods. Figure 4 The figure shows a comparison of the error curves for calculating the projectile pitch angle using the method of this invention and the distributed LSTM method. As can be seen from the figure, the error fluctuation range of the projectile pitch angle estimation using the method of this invention is controlled within ±0.8°, while the error fluctuation range of the projectile pitch angle estimation using the distributed LSTM method is within ±1°. The pitch angle estimation value of the method of this invention has a smaller error. Figure 5 The figure shows a comparison of the error curves for calculating the roll angle of a projectile using the method of this invention and the distributed LSTM method. As can be seen from the figure, the error fluctuation range of the projectile roll angle estimation using the method of this invention is controlled within ±1°, while the error fluctuation range of the projectile roll angle calculation using the distributed LSTM method is within ±1.5°. Therefore, the roll angle estimation value of the method of this invention has a smaller error. In summary... Figure 4 , Figure 5 It can be seen that the method of the present invention has higher estimation accuracy.

[0226] Figure 6 The figure shows the estimation time of a single frame image when using the method of this invention and the distributed LSTM method to estimate the attitude angle of the projectile. It can be seen from the figure that the estimation time of a single frame image of both is much smaller than the sampling time interval of 31.25 milliseconds. At the same time, the estimation time of a single frame image of the method of this invention is smaller than that of the distributed LSTM method. This shows that the method of this invention has good real-time performance.

[0227] Example 2

[0228] Based on the same inventive concept as Embodiment 1, this embodiment introduces a projectile attitude angle estimation device based on a spatiotemporal fusion dual-branch network, such as... Figure 7 As shown, it includes:

[0229] The preprocessing module is used to split and linearly transform the time-domain signal of the infrared focal plane array to obtain an infrared grayscale image sequence.

[0230] The pitch angle branch module is used to perform feature encoding and temporal modeling on infrared grayscale image sequences using the pitch angle branch of the spatiotemporal fusion dual-branch network to obtain temporally enhanced pitch angle features.

[0231] The roll angle branch module is used to perform feature encoding and temporal modeling on infrared grayscale image sequences using the roll angle branch of the spatiotemporal fusion dual-branch network to obtain temporally enhanced roll angle features.

[0232] The attitude angle calculation module is used to input the temporal enhanced pitch angle features and temporal enhanced roll angle features into the corresponding multilayer perceptrons to obtain the spatiotemporally fused pitch angle sine and cosine representations and roll angle sine and cosine representations. The four-quadrant arctangent function is used to calculate the pitch angle sine and cosine representations and roll angle sine and cosine representations respectively to obtain the estimated pitch angle and roll angle of the projectile.

[0233] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0234] Example 3

[0235] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the steps of the projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network described in Embodiment 1.

[0236] In summary, this invention reconstructs the temporal signal of an infrared focal plane array into a grayscale image. It fully mines the spatial structure and frequency energy features of a single-frame image through a spatiotemporal fusion dual-branch network, and combines this with a temporal convolutional network to mine cross-frame temporal correlations. This multi-dimensional information mining significantly improves the accuracy and real-time performance of projectile attitude estimation in dynamic scenes. Furthermore, considering the differentiated representation patterns of pitch and roll angles, this invention designs independent branch extraction paths, effectively avoiding feature confusion and ensuring the relevance and effectiveness of feature representation.

[0237] In the pitch angle branch, this invention integrates multi-scale frequency domain amplitude modeling and directional statistical enhancement to achieve deep synergy between spatial and frequency domain features. Then, by using a temporal convolutional network to fuse single-frame spatial-frequency domain features with multi-frame temporal dynamic features, it accurately captures the global energy distribution and spatiotemporal evolution of pitch angle changes, enabling pitch attitude estimation to maintain high stability in high-speed dynamic scenarios.

[0238] In the roll angle branch, this invention employs in-sample normalization to eliminate overall amplitude interference and combines bidirectional one-dimensional convolution of rows and columns to efficiently extract local directional structural features, significantly reducing computational complexity. Furthermore, it integrates spatial local features with temporal evolution information through a temporal convolutional network, thereby effectively improving the accuracy of roll attitude estimation while ensuring real-time performance.

[0239] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0240] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0241] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0242] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0243] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A projectile attitude angle estimation method based on a spatiotemporal fusion dual-branch network, characterized in that, include: The infrared grayscale image sequence is obtained by splitting and linearly transforming the time-domain signal of the infrared focal plane array. The pitch angle related spatial features of infrared grayscale image sequences are extracted by using the pitch angle branch of the spatiotemporal fusion dual-branch network, and temporal modeling is performed to obtain temporally enhanced pitch angle features; The roll angle related spatial features of infrared grayscale image sequences are extracted by utilizing the roll angle branch of the spatiotemporal fusion dual-branch network, and temporal modeling is performed to obtain temporally enhanced roll angle features. The temporal enhanced pitch angle feature and the temporal enhanced roll angle feature are respectively input into the corresponding multilayer perceptron to obtain the spatiotemporal fusion pitch angle sine and cosine representation results and roll angle sine and cosine representation results; By using the four-quadrant arctangent function to solve the sine and cosine representations of the pitch angle and the roll angle, respectively, the estimated values ​​of the projectile's pitch angle and roll angle are obtained.

2. The projectile attitude angle estimation method according to claim 1, characterized in that, The method involves extracting pitch angle-related spatial features from infrared grayscale image sequences using the pitch angle branch of a spatiotemporal fusion dual-branch network, and performing temporal modeling to obtain temporally enhanced pitch angle features, including: Multi-scale frequency domain amplitude modeling is performed on a single frame of infrared grayscale image in an infrared grayscale image sequence to obtain the amplitude spectrum at each scale; The amplitude spectra at each scale are input into a convolutional neural network, and high-dimensional features in the frequency domain are obtained through feature extraction. Row vector statistics are performed on a single frame of infrared grayscale image, and then the row vector statistical features are obtained by mapping using a multilayer perceptron. By splicing together high-dimensional features in the frequency domain and statistical features of row vectors, the spatial features related to the pitch angle of a single frame are obtained. Based on the spatial features related to the pitch angle of each frame in the infrared grayscale image sequence, a temporal feature sequence of pitch angle is constructed. The pitch angle temporal feature sequence is input into a pitch angle temporal convolution module composed of stacked multi-layer one-dimensional dilated convolutional residual blocks for temporal modeling, resulting in a pitch angle temporal enhanced feature sequence: ； in, After representing the timing model Temporal enhanced pitch angle feature sequence at time step, Indicates the first time series modeling Temporal enhancement features of pitch angle in frame images, The length of the infrared grayscale image sequence; Extract the last time step feature of the pitch angle temporal enhancement feature sequence as the temporal enhancement pitch angle feature. : 。 3. The projectile attitude angle estimation method according to claim 2, characterized in that, The step of performing multi-scale frequency domain amplitude modeling on a single frame of infrared grayscale image in an infrared grayscale image sequence to obtain the amplitude spectrum at each scale includes: Using a two-dimensional bilinear downsampling operator to convert a single frame of infrared grayscale image Constructing a multi-scale image: ； ； in, This is the image after downsampling when the scale factor is s. , This represents a two-dimensional bilinear downsampling operator with a scaling factor of s. This represents the image after downsampling when the scale factor is s. mid-coordinate point The pixel grayscale value at that location, and These represent the height and width of the image, respectively. Image before downsampling mid-coordinate point The pixel grayscale value at that location, The weights of the two-dimensional bilinear downsampling operator; Weight The calculation formula is: ； in, This is for calculating the maximum value. Perform two-dimensional fast Fourier transform on images at each scale: ； in, This represents the coordinates of the image with scale factor s after undergoing a two-dimensional fast Fourier transform. Frequency domain coefficients at that location, For frequency domain coordinates, The scale factor is represented as coordinates of the image The pixel grayscale value at that location, The imaginary unit; Define the amplitude spectrum of images at each scale: ； in, The scaling factor is The Fourier amplitude spectrum of the image. The scaling factor is The results of the two-dimensional fast Fourier transform of the image. , For complex number modulus operations; A nonlinear transformation of the amplitude spectrum is performed using a logarithmic mapping: ； in, The amplitude spectrum after nonlinear transformation. It is a logarithmic function.

4. The projectile attitude angle estimation method according to claim 2, characterized in that, The process of inputting the amplitude spectrum at each scale into a convolutional neural network and obtaining high-dimensional frequency domain features through feature extraction includes: Amplitude spectra at various scales Input convolutional neural network Extract frequency-space joint features at each scale. ; Frequency-spatial joint features of the original scale image Weighted processing is performed to obtain The formula is as follows: ； ； in, The original scale frequency domain features are weighted. For element-wise multiplication, This is the confidence plot corresponding to the original scale. This represents the Sigmoid activation function. This is a 1×1 convolution operation; Global average pooling is performed on features at each scale to obtain feature vectors at each scale. The formula is as follows: ； in, The feature vector corresponding to the scale factor s, The original scale frequency domain features after weighting mid-coordinate point eigenvalues ​​at that location scale factor Corresponding frequency-space joint features mid-coordinate point eigenvalues ​​at that location and These represent the height and width of the image, respectively. By concatenating feature vectors at various scales and fusing them through a fully connected layer, high-dimensional features in the frequency domain can be obtained. : ； in, It is a mapping module consisting of a fully connected layer and a nonlinear activation function.

5. The projectile attitude angle estimation method according to claim 2, characterized in that, The process of performing row vector statistics on a single frame of infrared grayscale image and then using a multilayer perceptron to map and obtain row vector statistical features includes: Calculate a single frame of infrared grayscale image row mean vector : ； ； in, Represents the first in an infrared grayscale image The mean of all elements in a row. Represents the first in an infrared grayscale image line, number Column pixel values, and These represent the height and width of the image, respectively. Using a multilayer perceptron to analyze the row mean vector Mapping yields row vector statistical features .

6. The projectile attitude angle estimation method according to claim 1, characterized in that, The method involves extracting roll angle-related spatial features from infrared grayscale image sequences using the roll angle branch of a spatiotemporal fusion dual-branch network, and performing temporal modeling to obtain temporally enhanced roll angle features, including: For a single frame of infrared grayscale image Intra-sample standardization is performed to obtain the standardized image. : ； in, To prevent extremely small constants with a denominator of zero, , Infrared grayscale images The mean and standard deviation; Standardized image Inputting into a column-oriented one-dimensional convolutional neural network, the column-oriented convolutional features are obtained through feature extraction. : ； in, The GELU nonlinear activation function is used. This represents a column-oriented one-dimensional convolution operator. and These represent the column-oriented one-dimensional convolution weights and biases, respectively. Column-wise convolution features One-dimensional global average pooling is used to obtain the column structure vector. : ； in, This represents a one-dimensional global average pooling operation. Represents column-wise convolutional features The Middle Feature values ​​at each position, Indicates the height of the image; Standardized image Transpose to get: ； in, This represents the transposed image; Transposed image Inputting into a row-oriented one-dimensional convolutional neural network, the row-oriented convolutional features are obtained through feature extraction. : ； in, This represents a row-oriented one-dimensional convolution operator. and These represent the row-directed one-dimensional convolution weights and biases, respectively. row-wise convolutional features One-dimensional global average pooling is used to obtain the row structure vector. : ； in, Representing row-wise convolutional features The Middle Feature values ​​at each position, Indicates the width of the image; Concatenate column structure vectors with row structure vector The spatial features related to the roll angle of a single frame are obtained. Based on the spatial features related to the roll angle of each frame in the infrared grayscale image sequence, a roll angle temporal feature sequence is constructed. The roll angle temporal feature sequence is input into a roll angle temporal convolutional module composed of stacked multi-layer one-dimensional dilated convolutional residual blocks for temporal modeling, resulting in a roll angle temporal enhanced feature sequence: ； in, After representing the timing model Temporal enhanced roll angle feature sequence at time step, Indicates the first time series modeling Temporal enhancement features of roll angle in frame images, The length of the infrared grayscale image sequence; Extract the last time step feature of the roll angle temporal enhancement feature sequence as the temporal enhancement roll angle feature. : 。 7. The projectile attitude angle estimation method according to claim 1, characterized in that, Enhance pitch angle features with time series Temporally enhanced roll angle features Input the corresponding multilayer perceptrons respectively , The sine and cosine representations of the attitude angles after spatiotemporal fusion are obtained as follows: ； in, The result is represented by the sine and cosine of the pitch angle at time t after spatiotemporal fusion. The result is represented by the sine and cosine of the roll angle at time t after spatiotemporal fusion.

8. The projectile attitude angle estimation method according to claim 1, characterized in that, The method of using the four-quadrant arctangent function to solve the sine and cosine representations of the pitch angle and roll angle, respectively, to obtain the estimated values ​​of the projectile's pitch angle and roll angle includes: Based on the sine and cosine representations of the pitch and roll angles, the calculated pitch and roll angle values ​​are obtained: ； in, and These are the calculated pitch angle and roll angle values ​​at time t, respectively. The arctangent function is in the fourth quadrant, and its output quadrant is... , and Representing the pitch angle, sine and cosine respectively, and representing the result. The sine and cosine components, and The results represent the roll angle (sine and cosine). The sine and cosine components; Pitch angle calculation value and roll angle calculation value Angle range conversion The estimated pitch angle of the projectile is obtained. and roll angle estimate : ； 。 9. A projectile attitude angle estimation device based on a spatiotemporal fusion dual-branch network, characterized in that, include: The preprocessing module is used to split and linearly transform the time-domain signal of the infrared focal plane array to obtain an infrared grayscale image sequence; The pitch angle branch module is used to extract pitch angle-related spatial features of infrared grayscale image sequences using the pitch angle branch of the spatiotemporal fusion dual-branch network, and to perform temporal modeling to obtain temporally enhanced pitch angle features; The roll angle branch module is used to extract roll angle-related spatial features of infrared grayscale image sequences using the roll angle branch of the spatiotemporal fusion dual-branch network, and perform temporal modeling to obtain temporally enhanced roll angle features. The attitude angle calculation module is used to input the temporal enhanced pitch angle features and temporal enhanced roll angle features into the corresponding multilayer perceptrons to obtain the spatiotemporally fused pitch angle sine and cosine representations and roll angle sine and cosine representations. The four-quadrant arctangent function is used to calculate the pitch angle sine and cosine representations and roll angle sine and cosine representations respectively to obtain the estimated pitch angle and roll angle of the projectile.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the projectile attitude angle estimation method based on spatiotemporal fusion dual-branch network as described in any one of claims 1-8.

Images