Radar infrared signal feature extraction and fusion method based on time sequence convolutional network

Abstract

The invention belongs to the technical field of radar and infrared signal processing, and particularly discloses a radar infrared signal feature extraction and fusion method based on a time sequence convolutional network, and the method comprises the steps: obtaining radar time frequency data and a radar distance-Doppler graph of a to-be-extracted feature, and obtaining an infrared image of the to-be-extracted feature; performing feature extraction on the radar time-frequency data by using a time sequence full-connection network to obtain radar time-frequency data features, and performing feature extraction on the radar distance-Doppler map and the infrared image by using a time sequence convolutional network to obtain radar distance-Doppler map features and infrared image features; taking the radar time-frequency data features as main features, and performing cross attention calculation on the main features, the radar distance-Doppler image features and the infrared image features to obtain intermediate features; and fusing the intermediate features to obtain fused features, and performing target identification by using the fused features. According to the scheme, the technical problem that the reliability and precision of existing radar infrared signal feature extraction and fusion are low is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of radar and infrared signal processing, and in particular relates to a radar infrared signal feature extraction and fusion method based on a temporal convolutional network. Background Art

[0002] In the new combat environment of the future, the combat objectives, environment, and tasks faced by precision-guided detection technology will change significantly. Strong ground clutter will be widely distributed, and advanced interference methods such as active interference and passive interference will be widely used. Many factors will make the detection and combat environment of precision-guided weapons increasingly harsh, and put forward higher requirements on the accuracy and robustness of target detection and recognition.

[0003] Traditional target detection methods have gone through a technical process from statistical pattern recognition, model-based, inference rule-based, shallow network learning-based, and deep learning-based. They usually use artificially designed features to enhance target signals and suppress clutter backgrounds for specific targets and environments. However, the information dimension and information processing capabilities of traditional artificially designed features are insufficient, the application scenarios are single, and the generalization performance is insufficient. In recent years, the booming artificial intelligence technology has the ability to extract high-dimensional features. However, deep learning technology based on convolutional neural networks has a strong dependence on data and annotations, and military data is generally scarce. When there is variable interference or random task input, it is often difficult to respond effectively.

[0004] On the other hand, due to its inherent weaknesses and limitations, precision-guided weapons with a single guidance mode severely limit their combat effectiveness and are difficult to cope with the complex and harsh battlefield environment in the future. This has prompted the development of seekers from single mode to dual mode and multi-mode. Information fusion method is one of the most important links in composite guidance information fusion. By using the information obtained by radar / infrared sensors, autonomous analysis and comprehensive optimization based on certain optimization rules can obtain new information that is more credible and accurate than a single sensor, eliminate useless or contradictory information, and make accurate decisions. Summary of the invention

[0005] The purpose of the present invention is to provide a radar infrared signal feature extraction and fusion method based on a temporal convolutional network to solve the technical problems of low reliability and precision in the existing radar infrared signal feature extraction and fusion.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A radar infrared signal feature extraction and fusion method based on a temporal convolutional network comprises the following steps: preprocessing a radar signal to obtain radar time-frequency data and a radar range-Doppler map of features to be extracted, and preprocessing an infrared signal to obtain an infrared image of features to be extracted; extracting features from the radar time-frequency data using a temporal fully connected network to obtain radar time-frequency data features, and extracting features from the radar range-Doppler map and the infrared image using a temporal convolutional network to obtain radar range-Doppler map features and infrared image features; taking the radar time-frequency data features as main features, and performing cross-attention calculations with the radar range-Doppler map features and the infrared image features to obtain intermediate features; fusing the intermediate features to obtain fused features, so as to use the fused features for target recognition.

[0008] Preferably, the radar signal is preprocessed to obtain radar time-frequency data of the features to be extracted, including: performing time-frequency transformation on the acquired radar echo signal to generate radar time-frequency data; and using a double trilinear interpolation method to scale the radar signal time-frequency diagram to match the above-mentioned timing fully connected network input.

[0009] Preferably, the radar signal is preprocessed to obtain a radar range-Doppler map of features to be extracted, including: generating a radar RD image based on the acquired radar echo signal; standardizing the radar RD image, and adjusting the resolution of the radar RD image to match the temporal convolutional network.

[0010] Preferably, before performing feature extraction, the method further includes: training the above-mentioned temporal convolutional network according to a training set.

[0011] Preferably, training the above-mentioned temporal convolutional network according to the training set includes: using the above-mentioned training set to train the above-mentioned temporal convolutional network through a random gradient descent algorithm and a step-by-step learning rate adjustment method.

[0012] Preferably, a temporal convolutional network is used to extract features from the radar range-Doppler map and the infrared image respectively to obtain radar range-Doppler map features and infrared image features, including: using the above-mentioned temporal convolutional network to perform at least one temporal convolution processing on the radar range-Doppler map and the infrared image respectively to obtain radar range-Doppler map features and infrared image features.

[0013] Preferably, the above-mentioned temporal convolution network is used to perform at least one temporal convolution processing on the radar range-Doppler map and the infrared image respectively, including: performing a first temporal convolution processing on the radar range-Doppler map and the infrared image to obtain a first temporal convolution vector; when performing a second temporal convolution processing, pooling the above-mentioned first temporal convolution vector to obtain a first intermediate feature vector; performing a temporal convolution processing on the above-mentioned first intermediate feature vector to obtain a second temporal convolution vector; pooling the above-mentioned second temporal convolution vector to obtain a second intermediate feature vector; performing a temporal convolution processing on the above-mentioned second intermediate feature vector to obtain a second temporal convolution vector; performing a temporal convolution processing on the above-mentioned second intermediate feature vector to obtain a second temporal convolution vector, and so on, performing a temporal convolution processing to obtain a first feature vector.

[0014] Preferably, after using the above-mentioned temporal convolution network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, it also includes: integrating and reducing the dimension of the first feature vector obtained by the temporal convolution process to obtain radar range-Doppler map features and infrared image features.

[0015] Preferably, the first feature vector obtained by the time series convolution processing is integrated and dimensionally reduced to obtain radar range-Doppler map features and infrared image features, including: integrating the above-mentioned first feature vector according to the three-dimensional pooling layer to obtain the second feature vector; and reducing the dimension of the above-mentioned second feature vector to obtain radar range-Doppler map features and infrared image features.

[0016] A radar infrared signal feature extraction and fusion system based on a temporal convolutional network comprises: a preprocessing unit, used for preprocessing a radar signal to obtain radar time-frequency data and a radar range-Doppler map of features to be extracted, and preprocessing an infrared signal to obtain an infrared image of features to be extracted; an extraction unit, used for extracting features from radar time-frequency data using a temporal fully connected network to obtain radar time-frequency data features, and respectively extracting features from radar range-Doppler maps and infrared images using a temporal convolutional network to obtain radar range-Doppler map features and infrared image features; a processing unit, used for taking the radar time-frequency data features as main features, and respectively performing cross-attention calculations with radar range-Doppler map features and infrared image features to obtain intermediate features; and a fusion unit, used for fusing the intermediate features to obtain fused features, so as to use the fused features for target recognition.

[0017] A computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the above-mentioned radar infrared signal feature extraction and fusion method based on a temporal convolutional network when running.

[0018] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the radar infrared signal feature extraction and fusion method based on a temporal convolutional network through the computer program.

[0019] In the present invention, radar time-frequency data and radar range-Doppler map of features to be extracted are obtained by preprocessing radar signals, and infrared signals are preprocessed to obtain infrared images of features to be extracted; radar time-frequency data are feature extracted by using a time series fully connected network to obtain radar time-frequency data features, and radar range-Doppler map and infrared image are feature extracted by using a time series convolutional network to obtain radar range-Doppler map features and infrared image features; radar time-frequency data features are used as main features, and cross-attention calculations are performed with radar range-Doppler map features and infrared image features to obtain intermediate features; intermediate features are fused to obtain fused features, and fused features are used to perform target recognition, thereby fully utilizing and capturing brain-like neuron membrane potential accumulation, action potential excitation, and biomembrane reset effects. The ability of dynamic process is used for high-precision and efficient time series information feature extraction, and the intrinsic time domain expression ability of neuron membrane potential in time series pulse convolution network and time series pulse fully connected network is fully utilized. The time domain information expression ability can be added with almost no additional parameters and only a very small amount of calculation, and the signal features can be extracted more accurately. Then, the multi-modal signal features are fused through the cross-attention mechanism, and multi-modal signal features are cross-fused to provide effective support for aircraft precision strikes, significantly reduce the computational complexity of network detection, and realize efficient feature extraction and recognition, improve target detection and recognition accuracy, solve the technical problems of low reliability and accuracy of existing radar infrared signal feature extraction and fusion, and achieve the technical effect of improving the reliability and accuracy of target precise tracking under complex interference confrontation conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a radar infrared signal feature extraction and fusion method based on a temporal convolutional network in an embodiment of the present invention;

[0021] Figure 2 It is a schematic diagram of an implementation method of a radar infrared signal feature extraction and fusion method based on a temporal convolutional network in an embodiment of the present invention;

[0022] Figure 3 It is a structural schematic diagram of a radar infrared signal feature extraction and fusion system based on a temporal convolutional network in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The advantages and features of the present invention will become more apparent from the following description and claims. It should be noted that the drawings are all in very simplified form and are not in precise proportions, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0024] It should be noted that, in order to clearly explain the content of the present invention, the present invention specifically cites multiple embodiments to further illustrate different implementations of the present invention, wherein the multiple embodiments are enumerated rather than exhaustive. In addition, for the sake of brevity of explanation, the contents mentioned in the previous embodiments are often omitted in the subsequent embodiments. Therefore, the contents not mentioned in the subsequent embodiments can refer to the previous embodiments accordingly.

[0025] Example 1

[0026] A radar infrared signal feature extraction and fusion method based on temporal convolutional network, such as Figure 1 As shown, the method includes:

[0027] S102, preprocessing the radar signal to obtain radar time-frequency data and radar range-Doppler map of features to be extracted, and preprocessing the infrared signal to obtain an infrared image of features to be extracted;

[0028] S104, using a time series fully connected network to extract features from the radar time-frequency data to obtain radar time-frequency data features, and using a time series convolutional network to extract features from the radar range-Doppler map and the infrared image to obtain radar range-Doppler map features and infrared image features;

[0029] S106, using the radar time-frequency data feature as the main feature, and performing cross-attention calculation with the radar range-Doppler map feature and the infrared image feature to obtain an intermediate feature;

[0030] S108, fusing the intermediate features to obtain fused features, and using the fused features to perform target recognition.

[0031] Optionally, the temporal fully connected network includes at least one temporal fully connected layer, and the temporal convolutional network includes at least one temporal convolutional layer.

[0032] As an optional implementation, the radar signal is preprocessed to obtain radar time-frequency data of the features to be extracted, including: performing time-frequency transformation on the acquired radar echo signal to generate radar time-frequency data; and using a double trilinear interpolation method to scale the radar signal time-frequency diagram to match the input of a time-series fully connected network.

[0033] As an optional implementation, the radar signal is preprocessed to obtain a radar range-Doppler map of the features to be extracted, including: generating a radar RD image based on the acquired radar echo signal; standardizing the radar RD image, and adjusting the resolution of the radar RD image to match the temporal convolutional network.

[0034] As an optional implementation, before feature extraction, the method further includes: training a temporal convolutional network according to a training set.

[0035] As an optional implementation, training the temporal convolutional network according to the training set includes: using the training set to train the temporal convolutional network through a random gradient descent algorithm and a step-by-step learning rate adjustment method.

[0036] As an optional implementation, a temporal convolutional network is used to extract features of a radar range-Doppler map and an infrared image respectively to obtain radar range-Doppler map features and infrared image features, including: using a temporal convolutional network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively to obtain radar range-Doppler map features and infrared image features.

[0037] As an optional implementation, a temporal convolution network is used to perform at least one temporal convolution processing on the radar range-Doppler map and the infrared image, respectively, including: performing a first temporal convolution processing on the radar range-Doppler map and the infrared image to obtain a first temporal convolution vector; when performing a second temporal convolution processing, pooling the first temporal convolution vector to obtain a first intermediate feature vector; performing a temporal convolution processing on the first intermediate feature vector to obtain a second temporal convolution vector; pooling the second temporal convolution vector to obtain a second intermediate feature vector; performing a temporal convolution processing on the second intermediate feature vector to obtain a second temporal convolution vector, and so on, performing a temporal convolution processing to obtain a first feature vector.

[0038] Optionally, the radar range-Doppler and infrared images to be extracted are subjected to at least one time series convolution process to obtain a first feature vector. Specifically, it is not limited to performing the first convolution process on the radar range-Doppler image and infrared image to be identified to obtain a first time series convolution vector, and determining the first time series convolution vector as the first feature vector; when performing n times of time series convolution process, the first time series convolution vector is pooled to obtain a first intermediate feature vector, the first intermediate feature vector is subjected to time series convolution process to obtain a second time series convolution vector, the second time series convolution vector is pooled to obtain a second intermediate feature vector, and so on, the nth time series convolution vector is determined as the first feature vector, and n is an integer greater than 1.

[0039] Optionally, the number of times of temporal convolution processing performed on the radar range-Doppler and infrared images whose features are to be extracted is not limited to being determined according to the temporal convolution layer structure in the temporal convolution network.

[0040] As an optional implementation, after using a temporal convolutional network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, it also includes: integrating and reducing the dimension of the first feature vector obtained by the temporal convolution process to obtain radar range-Doppler map features and infrared image features.

[0041] As an optional implementation, the first feature vector obtained by the time series convolution processing is integrated and dimensionally reduced to obtain radar range-Doppler map features and infrared image features, including: integrating the first feature vector according to a three-dimensional pooling layer to obtain a second feature vector; and reducing the dimension of the second feature vector to obtain radar range-Doppler map features and infrared image features.

[0042] The radar range-Doppler image and the infrared image to be extracted are subjected to at least one time series convolution process to obtain a first eigenvector, the first eigenvector is integrated to obtain a second eigenvector, the second eigenvector is subjected to dimensionality reduction to obtain a third eigenvector, and feature fusion is performed according to the third eigenvector.

[0043] Specifically, the radar infrared signal feature extraction and fusion method based on the temporal convolutional network is specifically implemented but not limited to the following: Figure 2 As shown, the following steps are included:

[0044] Step 1: preprocess the radar signal to obtain radar time-frequency data of features to be extracted;

[0045] Step 2, preprocessing the radar signal to obtain a radar range-Doppler map of the features to be extracted;

[0046] Step 3, preprocessing the infrared signal to obtain an infrared image of the features to be extracted;

[0047] Step 4, using a time series fully connected network to extract features of the radar signal time-frequency signal; wherein the time series fully connected network includes at least one time series fully connected layer;

[0048] Step 5, using a temporal convolutional network to extract features from the radar range-Doppler image and the infrared image respectively; wherein the temporal convolutional network includes at least one temporal convolutional layer;

[0049] Step 6: Take the radar time-frequency signal feature extracted in step 4 as the main feature, perform cross-attention calculation with the radar range-Doppler image feature and infrared image feature extracted in step 5, and add and fuse the intermediate features after the cross-attention calculation to obtain the fused feature;

[0050] Step 7: Use the fusion features obtained in step 6 to perform target recognition.

[0051] The temporal convolutional network is used to extract features of the radar range-Doppler map and the infrared map respectively, specifically including: performing at least one temporal convolution process on the radar range-Doppler map and the infrared map to be extracted to obtain a first feature vector; integrating the first feature vector according to a three-dimensional pooling layer to obtain a second feature vector; performing dimensionality reduction on the second feature vector to obtain a third feature vector; and performing feature fusion according to the third feature vector.

[0052] The radar range-Doppler map and the infrared map to be extracted features are subjected to at least one time series convolution process to obtain a first feature vector, including: performing a first convolution process on the radar range-Doppler map and the infrared map to be identified to obtain a first time series convolution vector; when performing the first time series convolution process, determining the first time series convolution vector as a first feature vector; when performing n time series convolution processes, performing pooling process on the first time series convolution vector to obtain a first intermediate feature vector, performing time series convolution process on the first intermediate feature vector to obtain a second time series convolution vector, performing pooling process on the second time series convolution vector to obtain a second intermediate feature vector, and so on, determining the nth time series convolution vector as the first feature vector, where n is an integer greater than 1.

[0053] Training the temporal convolutional network according to the training set specifically includes: training the temporal convolutional network by a random gradient descent algorithm and a step-by-step learning rate adjustment method.

[0054] The radar signal is preprocessed to obtain radar time-frequency data of the features to be extracted, including: obtaining the radar echo signal, performing time-frequency transformation, and generating radar time-frequency data; using the double trilinear interpolation method to scale the radar signal time-frequency diagram to a size that matches the convolutional pulse neural network input.

[0055] The radar signal is preprocessed to obtain the radar range-Doppler map of the features to be extracted, specifically including: obtaining the radar echo signal and generating the radar RD image; standardizing the radar RD image and adjusting the resolution to a size matching the temporal convolutional network.

[0056] The infrared signal is preprocessed to obtain an infrared image of the features to be extracted, specifically including: acquiring an infrared image; performing non-uniform correction and standardization preprocessing on the infrared image, and adjusting the resolution to a size matching the temporal convolutional network.

[0057] The temporal convolution layer is implemented using a two-dimensional convolutional spike neuron model. Furthermore, the temporal convolution layer is implemented using a two-dimensional convolutional leaky integral emission model.

[0058] The 2D convolutional leaky integral emission model is used for:

[0059] According to the two-dimensional convolution leaky integral distribution model input value X at time t t After convolution with weight W, we get value I t , and the biovoltage value at time t-1 After adding, the membrane potential value at time t is obtained

[0060] According to the membrane potential value at time t With the emission threshold V th , determine the output value F at time t t ;

[0061] According to the output value F at time t t Determine whether to reset the membrane potential and set the voltage V reset Determine the reset membrane potential value in,

[0062] According to the reset membrane potential value Determine the biovoltage value at time t in, α and β are the leakage factors of the Leak activation function;

[0063] Among them, the output value F at time t t As the input to the next layer of the cascaded 2D convolutional leaky integral-send model, the biological voltage value at time t As input for calculating the membrane potential value at time t+1.

[0064] According to the membrane potential value at time t and the emission threshold V th , determine the output value Ft at time t, specifically including:

[0065] If the membrane potential at time t is greater than or equal to the emission threshold V th , then determine the output value F at time t t is 1;

[0066] If the membrane potential at time t is less than the emission threshold V th , then determine the output value F at time t t is 0.

[0067] The cross attention calculation formula is as follows:

[0068]

[0069]

[0070] Among them, FSNN is the radar time-frequency feature extracted in step 4, F RD and F IR They are the radar range-Doppler image features and infrared image features extracted in step 5, and It is the intermediate feature to be fused after cross attention calculation.

[0071] The intermediate features after cross-attention calculation are added and fused to obtain the fused features. The specific feature fusion calculation formula is:

[0072]

[0073] In an embodiment of the present application, by making full use of the ability to capture the accumulation of brain-like neuronal membrane potential, action potential excitation and biological membrane reset effect, the dynamic process is used for high-precision and efficient timing information feature extraction, and the intrinsic time domain expression ability of the neuronal membrane potential in the timing pulse convolutional network and the timing pulse fully connected network is fully utilized. The time domain information expression ability can be added with almost no additional parameters and only a very small increase in calculation amount, and the signal features can be extracted more accurately. Then, the multimodal signal features are fused through the cross-attention mechanism, and multimodal signal features are cross-fused to provide effective support for aircraft precision strikes, significantly reduce the computational complexity of network detection, and realize efficient feature extraction and recognition, improve target detection and recognition accuracy, solve the technical problems of low reliability and accuracy of existing radar infrared signal feature extraction and fusion, and achieve the technical effect of improving the reliability and accuracy of target precise tracking under complex interference confrontation conditions.

[0074] Example 2

[0075] A radar infrared signal feature extraction and fusion system based on temporal convolutional network, such as Figure 3 As shown, the system includes:

[0076] A preprocessing unit 302 is used to preprocess the radar signal to obtain radar time-frequency data and radar range-Doppler map of the features to be extracted, and to preprocess the infrared signal to obtain an infrared image of the features to be extracted;

[0077] An extraction unit 304 is used to extract features from the radar time-frequency data using a time series fully connected network to obtain radar time-frequency data features, and to extract features from the radar range-Doppler map and the infrared image using a time series convolutional network to obtain radar range-Doppler map features and infrared image features;

[0078] The processing unit 306 is used to use the radar time-frequency data feature as the main feature, and perform cross-attention calculation with the radar range-Doppler map feature and the infrared image feature to obtain an intermediate feature;

[0079] The fusion unit 308 is used to fuse the intermediate features to obtain fused features, so as to use the fused features for target recognition.

[0080] Optionally, the preprocessing unit 302 preprocesses the radar signal to obtain radar time-frequency data of features to be extracted, including: performing time-frequency transformation on the acquired radar echo signal to generate radar time-frequency data; and using a bi-trilinear interpolation method to scale the radar signal time-frequency diagram to match the input of a time-series fully connected network.

[0081] Optionally, the preprocessing unit 302 preprocesses the radar signal to obtain a radar range-Doppler map of features to be extracted, including: generating a radar RD image based on the acquired radar echo signal; standardizing the radar RD image, and adjusting the resolution of the radar RD image to match the temporal convolutional network.

[0082] Optionally, the above-mentioned radar infrared signal feature extraction and fusion system based on the temporal convolutional network also includes a training unit, which is used to train the temporal convolutional network according to a training set before feature extraction.

[0083] Optionally, the training unit trains the temporal convolutional network according to the training set, including: using the training set to train the temporal convolutional network through a random gradient descent algorithm and a step-by-step learning rate adjustment method.

[0084] Optionally, the extraction unit 304 uses a temporal convolutional network to perform feature extraction on the radar range-Doppler map and the infrared image respectively to obtain radar range-Doppler map features and infrared image features, including: using the temporal convolutional network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively to obtain radar range-Doppler map features and infrared image features.

[0085] Optionally, the extraction unit 304 uses a temporal convolution network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, including: performing a first temporal convolution process on the radar range-Doppler map and the infrared image to obtain a first temporal convolution vector; when performing a second temporal convolution process, pooling the first temporal convolution vector to obtain a first intermediate feature vector; performing a temporal convolution process on the first intermediate feature vector to obtain a second temporal convolution vector; pooling the second temporal convolution vector to obtain a second intermediate feature vector, performing a temporal convolution process using the second intermediate feature vector to obtain a second temporal convolution vector, and so on, performing a temporal convolution process to obtain a first feature vector.

[0086] Optionally, after the extraction unit 304 uses the temporal convolution network to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, it also includes: integrating and reducing the dimension of the first feature vector obtained by the temporal convolution process to obtain radar range-Doppler map features and infrared image features.

[0087] Optionally, the extraction unit 304 integrates and performs dimensionality reduction processing on the first feature vector obtained by the time series convolution processing to obtain radar range-Doppler map features and infrared image features, including: integrating the first feature vector according to the three-dimensional pooling layer to obtain the second feature vector; performing dimensionality reduction processing on the second feature vector to obtain radar range-Doppler map features and infrared image features.

[0088] In an embodiment of the present application, radar time-frequency data and a radar range-Doppler map of features to be extracted are obtained by preprocessing the radar signal, and an infrared image of features to be extracted is obtained by preprocessing the infrared signal; a time-series fully connected network is used to extract features of the radar time-frequency data to obtain radar time-frequency data features, and a time-series convolutional network is used to extract features of the radar range-Doppler map and the infrared image, respectively, to obtain radar range-Doppler map features and infrared image features; the radar time-frequency data features are used as main features, and cross-attention calculations are performed with the radar range-Doppler map features and the infrared image features, respectively, to obtain intermediate features; the intermediate features are fused to obtain fused features, and the fused features are used for target recognition, thereby making full use of the brain-like neuron membrane potential accumulation, action potential excitation and biomembrane reset to capture the target. The effect is capable of using dynamic processes for high-precision and efficient time series information feature extraction, and the intrinsic time domain expression ability of neuron membrane potential in the time series pulse convolutional network and the time series pulse fully connected network is fully utilized. The time domain information expression ability can be added with almost no additional parameters and only a very small amount of calculation, and the signal features can be extracted more accurately. Then, the multi-modal signal features are fused through the cross-attention mechanism, and multi-modal signal features are cross-fused to provide effective support for aircraft precision strikes, significantly reduce the computational complexity of network detection, and realize efficient feature extraction and recognition, improve target detection and recognition accuracy, solve the technical problems of low reliability and accuracy of existing radar infrared signal feature extraction and fusion, and achieve the technical effect of improving the reliability and accuracy of target precise tracking under complex interference confrontation conditions.

[0089] Example 3

[0090] Another aspect of the embodiments of the present invention further provides an electronic device for implementing the above-mentioned radar infrared signal feature extraction and fusion method based on a temporal convolutional network, and the electronic device is not limited to being a terminal device or a server in a radar system. The electronic device is not limited to including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the steps in any of the above-mentioned method embodiments through the computer program.

[0091] Example 4

[0092] Another aspect of the embodiments of the present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional embodiments of the radar infrared signal feature extraction and fusion method based on the temporal convolutional network. The computer program is configured to execute the steps of any of the above method embodiments when running.

Claims

1. A radar infrared signal feature extraction and fusion method based on a temporal convolutional network, characterized in that: include: Preprocessing the radar signal to obtain radar time-frequency data and radar range-Doppler map of the features to be extracted, and preprocessing the infrared signal to obtain an infrared image of the features to be extracted; The radar time-frequency data is extracted using a time-series fully connected network to obtain the radar time-frequency data features, and the radar range-Doppler map and infrared image are extracted using a time-series convolutional network to obtain the radar range-Doppler map features and infrared image features. The radar time-frequency data feature is used as the main feature, and cross-attention calculation is performed with the radar range-Doppler map feature and the infrared image feature to obtain an intermediate feature; The intermediate features are fused to obtain fused features, so as to use the fused features for target recognition.

2. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 1, characterized in that: Preprocess the radar signal to obtain radar time-frequency data for feature extraction, including: Perform time-frequency transformation on the acquired radar echo signal to generate radar time-frequency data; The radar signal time-frequency diagram is scaled to match the input of the temporal fully connected network using bi-trilinear interpolation.

3. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 1, characterized in that: Preprocess the radar signal to obtain the radar range-Doppler map of the features to be extracted, including: Generate a radar RD image based on the acquired radar echo signal; The radar RD image is standardized and its resolution is adjusted to match the temporal convolutional network.

4. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 1, characterized in that: Before feature extraction, it also includes: The temporal convolutional network is trained according to the training set.

5. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 4, characterized in that: Training the temporal convolutional network according to the training set includes: The training set is used to train the temporal convolutional network through a random gradient descent algorithm and a step-by-step learning rate adjustment method.

6. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 1, characterized in that: The temporal convolutional network is used to extract features of the radar range-Doppler map and infrared image respectively, and the radar range-Doppler map features and infrared image features are obtained, including: The temporal convolution network is used to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, so as to obtain the radar range-Doppler map features and the infrared image features.

7. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 6, characterized in that: The temporal convolution network is used to perform at least one temporal convolution process on the radar range-Doppler map and the infrared image respectively, including: Performing a first time series convolution process on the radar range-Doppler map and the infrared image to obtain a first time series convolution vector; When performing the second time series convolution processing, performing pooling processing on the first time series convolution vector to obtain a first intermediate feature vector; Performing time series convolution processing on the first intermediate feature vector to obtain a second time series convolution vector; Pooling is performed on the second time series convolution vector to obtain a second intermediate feature vector, and time series convolution processing is performed using the second intermediate feature vector to obtain a second time series convolution vector. Time series convolution processing is performed in this way to obtain a first feature vector.

8. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 7, characterized in that: After performing at least one time-series convolution process on the radar range-Doppler map and the infrared image respectively using the time-series convolution network, the method further includes: The first eigenvector obtained by the time series convolution process is integrated and dimensionally reduced to obtain the radar range-Doppler map features and infrared image features.

9. The radar infrared signal feature extraction and fusion method based on a temporal convolutional network as claimed in claim 8, characterized in that: The first feature vector obtained by time series convolution is integrated and dimensionally reduced to obtain radar range-Doppler map features and infrared image features, including: Performing integration processing on the first feature vector according to a three-dimensional pooling layer to obtain a second feature vector; The second feature vector is subjected to dimensionality reduction processing to obtain radar range-Doppler map features and infrared image features.

10. A radar infrared signal feature extraction and fusion system based on a temporal convolutional network, characterized in that: include: A preprocessing unit is used to preprocess the radar signal to obtain radar time-frequency data and radar range-Doppler map of the features to be extracted, and to preprocess the infrared signal to obtain an infrared image of the features to be extracted; An extraction unit is used to extract features from radar time-frequency data using a time-series fully connected network to obtain radar time-frequency data features, and to extract features from radar range-Doppler maps and infrared images using a time-series convolutional network to obtain radar range-Doppler map features and infrared image features; A processing unit, configured to use the radar time-frequency data feature as a main feature, and respectively perform cross-attention calculation with the radar range-Doppler map feature and the infrared image feature to obtain an intermediate feature; The fusion unit is used to fuse the intermediate features to obtain fused features, so as to use the fused features to perform target recognition.

Images