Intelligent target fusion recognition method and system based on radar multimodal data

Abstract

The present invention discloses an intelligent target fusion recognition method and system based on radar multimodal data, comprising: obtaining radar multimodal data of a target to be identified; inputting the radar multimodal data of the target to be identified into a trained multimodal radar data target fusion recognition model, extracting single-mode features in sequence, respectively obtaining multiple single-mode features, performing inter-modal similarity features and intrinsic feature learning based on multimodal feature learning subspace constraints according to the multiple single-mode features, compressing the learned multiple inter-modal similarity features to obtain multimodal similarity features, performing attention fusion on the learned multiple intrinsic features and multimodal similarity features, and outputting recognition probability to obtain the classification recognition result of the target in the radar multimodal data of the target to be identified. The present invention can efficiently and robustly identify the target.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar signal processing, and in particular relates to an intelligent target fusion recognition method and system based on radar multimodal data. Background Art

[0002] The radar broadband high-resolution range profile (HRRP) is the vector and amplitude waveform of the target scattering point echo projected on the radar ray obtained by the broadband radar signal. HRRP reflects the distribution of the main scattering center of the target along the radar line of sight at a certain radar viewing angle. Due to the advantages of easy acquisition, flexibility, and easy processing, HRRP is widely used in the field of aerial target recognition. However, due to the increasing complexity of actual application scenarios, target recognition based on a single HRRP cannot meet the needs of robust target recognition. At this time, multi-mode radar data fusion is based on multiple radar echo data, which can capture the complementary characteristics between different forms of radar data, thereby obtaining a complete description of the target.

[0003] In the prior art, in the process of using multimodal radar data for target recognition, there are problems such as a simple multimodal radar data fusion network structure, which does not fully exploit the complementary information contained in the multimodal data; there are also many data modalities, including both radar data and other data, such as optical data, which makes data acquisition requirements high and affects subsequent processing processes; there is also a lack of analysis of the feature relationship of multimodal data, and the fusion strategy is simple, making it difficult to fully utilize the information contained in data of different modalities.

[0004] Therefore, there is an urgent need to provide a target fusion recognition method for radar multimodal data to overcome the above defects. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides an intelligent target fusion recognition method and system based on radar multimodal data. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] In a first aspect, the present invention provides an intelligent target fusion recognition method based on radar multimodal data, comprising:

[0007] Acquire radar multi-mode data of the target to be identified;

[0008] Input the radar multimode data of the target to be identified into the trained multimode radar data target fusion recognition model, extract single-mode features in sequence, obtain multiple single-mode features respectively, perform inter-modal similarity features and intrinsic features learning based on multi-modal feature learning subspace constraints according to the multiple single-mode features, compress the learned multiple inter-modal similarity features to obtain multi-modal similarity features, perform attention fusion on the learned multiple intrinsic features and multi-modal similarity features, and output the recognition probability to obtain the classification recognition result of the target in the radar multimode data of the target to be identified;

[0009] Among them, the trained multi-mode radar data target fusion recognition model uses preset category data as the training data set to train the initial multi-mode radar data target fusion recognition model; the preset category data includes radar broadband high-resolution range image echo, narrowband modulation spectrum echo and target track sequence.

[0010] In a second aspect, the present invention further provides an intelligent target fusion recognition system based on radar multimodal data, comprising:

[0011] A data acquisition module, used to acquire radar multi-mode data of the target to be identified;

[0012] The data processing module is used to input the radar multi-mode data of the target to be identified into the trained multi-mode radar data target fusion recognition model, extract single-mode features in sequence, obtain multiple single-mode features respectively, perform inter-modal similarity feature and intrinsic feature learning based on the multi-modal feature learning subspace constraint according to the multiple single-mode features, compress the learned multiple inter-modal similarity features to obtain multi-modal similarity features, perform attention fusion on the learned multiple intrinsic features and multi-modal similarity features, and output the recognition probability to obtain the classification recognition result of the target in the radar multi-mode data of the target to be identified;

[0013] Among them, the trained multi-mode radar data target fusion recognition model uses preset category data as the training data set to train the initial multi-mode radar data target fusion recognition model; the preset category data includes radar broadband high-resolution range image echo, narrowband modulation spectrum echo and target track sequence.

[0014] Beneficial effects of the present invention:

[0015] The present invention provides an intelligent target fusion recognition method and system based on radar multimodal data. The multimodal radar data is used to train the recognition model. The multimodal radar data includes radar broadband high-resolution range image echo, narrowband modulation spectrum echo, and target track sequence. The multimodal radar data can fully explore the target scattering center distribution characteristics, micro-Doppler characteristics and motion characteristics, and realize the multi-dimensional and multi-domain description of the target attributes. The cross-modal inter-modal similarity feature learning subspace and the intra-modal intrinsic feature learning subspace unique to each modality are adopted, and the effective learning of different types of multimodal features is guided by corresponding loss constraints, so that the feature learning between modalities and within modalities is strengthened, and the description ability of the features to the target attributes is improved. The adaptive feature fusion method is adopted to improve the effectiveness of feature fusion.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of an intelligent target fusion recognition method based on radar multimodal data provided by an embodiment of the present invention;

[0018] Figure 2 It is a schematic diagram of a trained multi-mode radar data target fusion recognition model provided by an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of a feature extraction module provided by an embodiment of the present invention;

[0020] Figure 4 It is a schematic diagram of a multi-modal feature learning subspace constraint module provided by an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of an inter-modality similarity feature compression layer provided by an embodiment of the present invention;

[0022] Figure 6 is a schematic diagram of a feature reconstruction module provided by an embodiment of the present invention;

[0023] Figure 7 It is a schematic diagram of the attention feature fusion module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0025] See also Figure 1 , Figure 1: is a flow chart of an intelligent target fusion recognition method based on radar multimodal data provided by an embodiment of the present invention. The intelligent target fusion recognition method based on radar multimodal data provided by the present invention includes:

[0026] S101. Acquire radar multi-mode data of a target to be identified.

[0027] Specifically, in this embodiment, the radar multimode data of the target to be identified is three types of aircraft target data, consisting of a jet aircraft, a propeller aircraft and a helicopter. Each data sample contains a radar broadband high-resolution range image echo with a dimension of 512; a narrowband modulation spectrum echo with a dimension of 1024; a target track sequence, which includes 7 moments, and each moment includes 6-dimensional measurement information, namely, distance, azimuth, altitude, attitude, speed and radar cross-sectional area.

[0028] S102, inputting the radar multimode data of the target to be identified into the trained multimode radar data target fusion recognition model, extracting single-mode features in sequence to obtain multiple single-mode features respectively, performing inter-modal similarity feature and intrinsic feature learning based on multi-modal feature learning subspace constraints according to the multiple single-mode features, compressing the learned multiple inter-modal similarity features to obtain multi-modal similarity features, performing attention fusion on the learned multiple intrinsic features and multi-modal similarity features, and outputting the recognition probability to obtain the classification recognition result of the target in the radar multimode data of the target to be identified.

[0029] Among them, the trained multi-mode radar data target fusion recognition model uses preset category data as the training data set to train the initial multi-mode radar data target fusion recognition model; the preset category data includes radar broadband high-resolution range image echo, narrowband modulation spectrum echo and target track sequence.

[0030] Specifically, in this embodiment, see Figure 2 , Figure 2 is a schematic diagram of a trained multi-mode radar data target fusion recognition model provided by an embodiment of the present invention. Before the initial multi-mode radar data target fusion recognition model is trained, it also includes:

[0031] Construct a multi-mode radar data target fusion recognition model; among them,

[0032] The multi-mode radar data target fusion recognition model includes multiple single-mode feature extraction modules, a multi-mode feature learning subspace constraint module, a similarity feature compression module, a feature reconstruction module and an attention feature fusion module; the single-mode feature extraction module is used to extract single-mode features, the multi-mode feature learning subspace constraint module includes an inter-modal similarity feature encoder and an intrinsic feature encoder, the inter-modal similarity feature encoder is used to encode all single-mode features to obtain multiple inter-modal similarity features, the intrinsic feature encoder is used to encode the corresponding single-mode features to obtain intrinsic features, the similarity feature compression module is used to compress multiple inter-modal similarity features to obtain multi-modal similarity features, the feature reconstruction module includes multiple decoders, which are used to reconstruct the corresponding single-mode features according to the multi-modal similarity features and the corresponding intrinsic features to obtain the loss of reconstructing each single-mode feature, the attention feature fusion module includes multiple attention fusion layers and classification layers, the attention fusion layer is used to fuse the multi-modal similarity features and the intrinsic features, and the classification layer is used to classify the fused features and output the recognition probability.

[0033] In this embodiment, the preset category of data is used as a training data set to train the initial multi-mode radar data target fusion recognition model, including:

[0034] Acquire a training data set, the training data includes a plurality of training samples, each training sample includes a radar broadband high-resolution range image echo, a narrowband modulation spectrum echo and a target track sequence;

[0035] Optionally, the total number of samples in the training data set is 1800, of which 600 are jet samples, 600 are propeller samples, and 600 are helicopter samples. The total number of samples in the test data set is 900, of which 300 are jet samples, 300 are propeller samples, and 300 are helicopter samples;

[0036] Label the training samples in the training data set to obtain the true labels of the training samples;

[0037] The training data set and the real labels of the training samples in the training data set are input into the multi-mode radar data target fusion recognition model for iterative training until the preset conditions are met, thereby obtaining a trained multi-mode radar data target fusion recognition model;

[0038] Optional, the batch size is 64 during training and the initial learning rate is , the optimizer is AdamW, and the total number of iterations is set to 100.

[0039] In this embodiment, the training data set and the real labels of the training samples in the training data set are input into the initial multi-mode radar data target fusion recognition model for iterative training, including:

[0040] For the In the first iterative training process, the training data set is input into the multi-mode radar data target fusion recognition model, and the single-mode feature extraction module is used to extract features, thereby obtaining radar broadband high-resolution range image echo features, narrow-band modulation spectrum echo features, and target track sequence features.

[0041] Input the radar broadband high-resolution range image echo features, narrowband modulation spectrum echo features and target track sequence features into the multi-mode feature learning subspace constraint module for processing, and obtain the similarity features between the radar broadband high-resolution range image echo modes, the similarity features between the narrowband modulation spectrum echo modes, the similarity features between the target track sequence modes, the radar broadband high-resolution range image echo intrinsic features, the narrowband modulation spectrum echo intrinsic features and the target track sequence intrinsic features respectively; according to the similarity features between the radar broadband high-resolution range image echo modes, the similarity features between the narrowband modulation spectrum echo modes and the similarity features between the target track sequence modes, calculate the consistency loss; according to the similarity features between the radar broadband high-resolution range image echo modes, the similarity features between the narrowband modulation spectrum echo modes, the similarity features between the target track sequence modes, the radar broadband high-resolution range image echo intrinsic features, the narrowband modulation spectrum echo intrinsic features and the target track sequence intrinsic features, calculate the difference loss;

[0042] The similarity features between radar broadband high-resolution range image echo modes, the similarity features between narrowband modulation spectrum echo modes and the similarity features between target track sequence modes are input into the similarity feature compression module for feature compression to obtain multi-mode similarity features;

[0043] Input the multi-mode similarity features, radar broadband high-resolution range image echo intrinsic features, narrow-band modulation spectrum echo intrinsic features, target track sequence intrinsic features, radar broadband high-resolution range image echo features, narrow-band modulation spectrum echo features and target track sequence features into the feature reconstruction module for processing to calculate the reconstruction loss;

[0044] The multi-mode similarity features, radar broadband high-resolution range image echo intrinsic features, narrowband modulation spectrum echo intrinsic features, target track sequence intrinsic features and the true labels of training samples in the training data set are input into the attention feature fusion module for processing to obtain the classification results and calculate the classification loss.

[0045] In this embodiment, the preset condition includes minimizing the sum of consistency loss, difference loss, reconstruction loss and classification loss during a finite number of iterative training processes.

[0046] In this embodiment, see Figure 3 , Figure 3: is a schematic diagram of a feature extraction module provided by an embodiment of the present invention. The process of acquiring radar broadband high-resolution range image echo features, narrowband modulation spectrum echo features and target track sequence features includes:

[0047] The training data set is input into the radar broadband high-resolution range image echo feature extraction module, and normalization and alignment operations are performed in sequence, and then feature extraction is performed to obtain the radar broadband high-resolution range image echo features;

[0048] Optionally, the radar broadband high-resolution range image echo is normalized and preprocessed by center-of-gravity alignment using L2 norm, and then a 3-layer convolutional neural network (CNN) and a 1-layer linear layer are used as extraction modules for feature extraction. The number of channels of each layer of convolutional neural network is 8, 16, and 32, respectively. The one-dimensional convolution kernel size in the convolutional neural network is 5, the convolution step size is 2, and the number of input and output nodes of the linear layer is (4096, 1024) respectively.

[0049] The training data set is input into the narrowband modulation spectrum echo feature extraction module, and clutter removal, main body compensation and normalization are performed in sequence, and then feature extraction is performed to obtain the narrowband modulation spectrum echo features;

[0050] Optionally, the narrowband modulation spectrum echo is preprocessed by removing clutter, subject compensation and normalization, and then a 3-layer convolutional neural network (CNN) and a 1-layer linear layer are used as extraction modules for feature extraction, the number of channels of each layer of the convolutional neural network is 8, 16, and 32 respectively, the one-dimensional convolution kernel size in the convolutional neural network is 5, the convolution step size is 2, and the number of input and output nodes of the linear layer is (8192, 1024) respectively;

[0051] The training data set is input into the target track sequence feature extraction module, and the format is reorganized and normalized in turn, and then feature extraction is performed to obtain the target track sequence features;

[0052] Optionally, the target track sequence is reorganized in a 7×6 matrix, preprocessed by mean variance normalization, and then a 3-layer Bi-LSTM and a 1-layer linear layer are used as extraction modules for feature extraction. The Bi-LSTM hidden layer size is 128, and the number of input and output nodes of the linear layer are (1792, 1024) respectively.

[0053] It can be understood that the single-mode feature extraction module includes a radar broadband high-resolution range image echo feature extraction module, a narrowband modulation spectrum echo feature extraction module and a target track sequence feature extraction module.

[0054] In this embodiment, see Figure 4 , Figure 41 is a schematic diagram of a multi-mode feature learning subspace constraint module provided by an embodiment of the present invention, wherein radar broadband high-resolution range image echo features, narrowband modulation spectrum echo features and target track sequence features are input into the multi-mode feature learning subspace constraint module for processing, including:

[0055] Inputting radar broadband high-resolution range image echo features, narrow-band modulation spectrum echo features and target track sequence features into an inter-modal similarity feature encoder for processing, obtaining radar broadband high-resolution range image echo inter-modal similarity features, narrow-band modulation spectrum echo inter-modal similarity features and target track sequence inter-modal similarity features;

[0056] Inputting the radar broadband high-resolution range image echo feature into the radar broadband high-resolution range image echo intrinsic feature encoder for processing to obtain the radar broadband high-resolution range image echo intrinsic feature;

[0057] Inputting the narrowband modulation spectrum echo feature into the narrowband modulation spectrum echo intrinsic feature encoder for processing to obtain the narrowband modulation spectrum echo intrinsic feature;

[0058] Inputting the target track sequence characteristics into the target track sequence intrinsic characteristic encoder for processing to obtain the target track sequence intrinsic characteristics;

[0059] Among them, the single-mode feature encoder includes a radar broadband high-resolution range image echo intrinsic feature encoder, a narrowband modulation spectrum echo intrinsic feature encoder and a target track sequence intrinsic feature encoder;

[0060] The consistency loss is calculated based on the similarity characteristics between radar broadband high-resolution range image echo modes, the similarity characteristics between narrowband modulation spectrum echo modes, and the similarity characteristics between target track sequence modes. The expression is:

[0061] ;

[0062] ;

[0063] in, represents the consistency loss, represents the feature vector, Representing modality and The corresponding inter-modality similarity feature pairs, Represents the similarity feature between modalities, It represents the radar broadband high-resolution range image echo mode. represents the narrowband modulation spectrum echo mode, represents the target track sequence mode, express The central moment difference, Indicates the maximum order of the calculated center distance, Express expectations, express The order central moment, Indicates the value range of the feature;

[0064] The difference loss is calculated based on the similarity characteristics between the radar broadband high-resolution range image echo modes, the similarity characteristics between the narrowband modulation spectrum echo modes, the similarity characteristics between the target track sequence modes, the radar broadband high-resolution range image echo intrinsic characteristics, the narrowband modulation spectrum echo intrinsic characteristics and the target track sequence intrinsic characteristics. The expression is:

[0065] ;

[0066] in, represents the difference loss, Indicates different modes, Representing modality The corresponding inter-modality similarity features, Representing modality The corresponding intrinsic features, represents the Frobenius norm, Representing modality The corresponding intrinsic features, Representing modality The corresponding intrinsic features, It represents the radar broadband high-resolution range image echo mode. represents the narrowband modulation spectrum echo mode, represents the target track sequence mode, Indicates transpose.

[0067] Optionally, the inter-modal similarity feature encoder, the radar broadband high-resolution range image echo intrinsic feature encoder, the narrowband modulation spectrum echo intrinsic feature encoder and the target track sequence intrinsic feature encoder are all composed of two linear layers, and the number of input nodes and the number of output nodes of each layer are (1024, 512) and (512, 512) respectively.

[0068] Optional, see Figure 5 , Figure 5 It is a schematic diagram of the inter-modal similarity feature compression layer provided by an embodiment of the present invention. The inter-modal similarity feature compression layer is composed of 2 linear layers, and the number of input and output nodes of each layer is (1536, 512) and (512, 512) respectively.

[0069] In this embodiment, see Figure 6 , Figure 61 is a schematic diagram of a feature reconstruction module provided by an embodiment of the present invention, wherein multi-mode similarity features, radar broadband high-resolution range image echo intrinsic features, narrow-band modulation spectrum echo intrinsic features, target track sequence intrinsic features, radar broadband high-resolution range image echo features, narrow-band modulation spectrum echo features and target track sequence features are input into the feature reconstruction module for processing, including:

[0070] Input the multi-mode similarity feature and the radar broadband high-resolution range image echo intrinsic feature into the radar broadband high-resolution range image echo feature decoder to obtain the reconstructed radar broadband high-resolution range image echo feature; calculate the radar broadband high-resolution range image echo feature reconstruction loss according to the reconstructed radar broadband high-resolution range image echo feature and the radar broadband high-resolution range image echo feature;

[0071] Inputting the multi-mode similarity feature and the narrowband modulation spectrum echo intrinsic feature into the narrowband modulation spectrum echo feature decoder to obtain the reconstructed narrowband modulation spectrum echo feature; calculating the narrowband modulation spectrum echo feature reconstruction loss according to the reconstructed narrowband modulation spectrum echo feature and the narrowband modulation spectrum echo feature;

[0072] Input the multi-mode similarity features and the target track sequence intrinsic features into the target track sequence feature decoder to obtain the reconstructed target track sequence features; calculate the target track sequence feature reconstruction loss according to the reconstructed target track sequence features and the target track sequence features;

[0073] The decoder includes a radar broadband high-resolution range image echo feature decoder, a narrowband modulation spectrum echo feature decoder and a target track sequence feature decoder;

[0074] Among them, the expression of radar broadband high-resolution range image echo feature reconstruction loss, narrowband modulation spectrum echo feature reconstruction loss or target track sequence feature reconstruction loss is:

[0075] ;

[0076] in, represents the reconstruction loss, Indicates different modes, represents the mean square error, Represents the modality extracted by the feature extractor Features, Represents the reconstructed mode Features, It represents the radar broadband high-resolution range image echo mode. represents the narrowband modulation spectrum echo mode, represents the target track sequence mode, represents the Frobenius norm.

[0077] Optionally, the radar broadband high-resolution range profile echo feature decoder, the narrowband modulation spectrum echo feature decoder and the target track sequence feature decoder are all composed of 2 linear layers, and the number of input and output nodes of each layer is (512, 512) and (512, 1024) respectively.

[0078] In this embodiment, see Figure 7 , Figure 7 is a schematic diagram of an attention feature fusion module provided by an embodiment of the present invention, wherein multi-mode similarity features, radar broadband high-resolution range image echo intrinsic features, narrowband modulation spectrum echo intrinsic features, target track sequence intrinsic features and true labels of training samples in a training data set are input into the attention feature fusion module for processing, including:

[0079] The multi-mode similarity features, radar broadband high-resolution range image echo intrinsic features, narrowband modulation spectrum echo intrinsic features and target track sequence intrinsic features are sequentially feature mapped and position encoded, and then input into the multi-layer attention fusion layer for feature fusion processing, and then input into the classification layer for classification to obtain the classification result; the classification loss is calculated based on the classification result and the true label of the training sample in the training data set, and its expression is:

[0080] ;

[0081] in, represents the classification loss, represents the total number of samples, represents the sample index, represents the true label, Represents the predicted label.

[0082] Optionally, the attention feature fusion module includes 1 attention fusion layer, the feature mapping dimension is 128, the classification token dimension is 128, the position encoding dimension is 128, the multi-mode attention contains 6 attention heads, each attention head contains 3 linear layers, the number of input and output nodes of each layer is (128, 128) respectively, the feedforward layer contains 2 linear layers, the number of input and output nodes of each layer is (128, 512) and (512, 128) respectively, and the classification layer is composed of 2 linear layers, and the number of input and output nodes of each layer is (128, 128) and (128, 3) respectively.

[0083] In summary, the intelligent target fusion recognition method based on radar multimodal data provided by the present invention has the following beneficial effects:

[0084] First, compared with the radar target recognition method that only uses a single mode, the present invention utilizes multi-mode radar data, including radar broadband high-resolution range image echo, narrowband modulation spectrum echo, and target track sequence, fully exploiting the target scattering center distribution characteristics, micro-Doppler characteristics, and motion characteristics, and can achieve a multi-dimensional and multi-domain description of target attributes.

[0085] Second, the intelligent target fusion recognition method and system based on radar multimodal data constructed by the present invention take into account the modal heterogeneity between different types of radar data; in order to reduce the impact of the difficulty in feature representation and inaccurate target description of heterogeneous and heterogeneous multimodal radar data on multimodal learning, a cross-modal inter-modal similarity feature learning subspace and a modal intra-modal intrinsic feature learning subspace unique to each modality are designed, and the model is guided to effectively learn different types of multimodal features through corresponding loss constraints; by dividing the multimodal feature learning subspace and loss constraints, the model's learning of inter-modal and intra-modal features is strengthened, and the feature's ability to describe target attributes is improved.

[0086] Third, the intelligent target fusion recognition method and system based on radar multimodal data constructed by the present invention designs a multimodal feature fusion module based on the attention mechanism. Compared with the conventional fully connected fusion network, it can perform adaptive feature fusion according to the quality of different modal data and the information contained therein, thereby improving the effectiveness of feature fusion.

[0087] In an optional embodiment of the present invention, the effect of the intelligent target fusion recognition method based on radar multimodal data provided in the above embodiment is verified by simulation experiments, specifically:

[0088] 1. Simulation conditions

[0089] The simulation experimental conditions of this embodiment are: the processor is Intel(R) i7 CPU, the main frequency is 3.20GHz, the memory capacity is 64GB, the graphics card is Nvidia RTX 3090, and the video memory capacity is 24GB.

[0090] The software platforms for the simulation experiment in this embodiment are: Python 3.8, Pytorch 2.0.

[0091] 2. Simulation content and result analysis

[0092] In the simulation experiment of the present invention, the method provided by the above embodiment and an existing technology are used to conduct target recognition experiments on three types of aircraft, and the recognition rates are compared. The existing technology refers to the paper "Deepresidual learning for image recognition" published by He, Kaiming et al., which applies a deep residual convolutional neural network to the recognition model. The original network is designed for two-dimensional image data. In this simulation experiment, its main structure is retained and the two-dimensional convolution layer is changed to a one-dimensional convolution layer to adapt to HRRP data. The recognition rate is introduced to evaluate the performance of the recognition model. The recognition rate is defined as the ratio of the number of correctly recognized samples to the total number of test samples. The higher the recognition rate, the more correctly recognized samples there are, and the better the recognition performance of the model.

[0093] Table 1 Performance comparison of the method of the present invention and existing target recognition technology

[0094]

[0095] By comparing the recognition rates of the present invention and the prior art, it can be seen that the present invention can significantly improve the recognition rate.

[0096] Based on the same inventive concept, the present invention also provides an intelligent target fusion recognition system based on radar multimodal data, which is used to implement the intelligent target fusion recognition method provided by the above embodiment of the present invention. The embodiment of the method is referred to above and will not be repeated here. The system includes:

[0097] A data acquisition module, used to acquire radar multi-mode data of the target to be identified;

[0098] A data processing module is used to input the radar multimode data of the target to be identified into the trained multimode radar data target fusion recognition model, extract single-mode features in sequence, obtain multiple single-mode features respectively, perform inter-modal similarity feature and intrinsic feature learning based on multi-modal feature learning subspace constraints according to the multiple single-mode features, compress the learned multiple inter-modal similarity features to obtain multi-modal similarity features, perform attention fusion on the learned multiple intrinsic features and multi-modal similarity features, and output the recognition probability to obtain the classification recognition result of the target in the radar multimode data of the target to be identified;

[0099] Among them, the trained multi-mode radar data target fusion recognition model uses preset category data as the training data set to train the initial multi-mode radar data target fusion recognition model; the preset category data includes radar broadband high-resolution range image echo, narrowband modulation spectrum echo and target track sequence.

[0100] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant is intended to cover non-exclusive inclusion, so that the article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the article or device including the elements. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The orientation or position relationship indicated by "up", "down", "left", "right", etc. is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0101] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification.

[0102] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. An intelligent target fusion recognition method based on radar multimodal data, characterized in that: include: Acquire radar multi-mode data of the target to be identified; Inputting the radar multimode data of the target to be identified into the trained multimode radar data target fusion recognition model, extracting single-mode features in sequence to obtain a plurality of single-mode features respectively, performing inter-modal similarity features and intrinsic feature learning based on multi-modal feature learning subspace constraints according to the plurality of single-mode features, compressing the learned multiple inter-modal similarity features to obtain multi-modal similarity features, performing attention fusion on the learned multiple intrinsic features and the multi-modal similarity features, and outputting the recognition probability to obtain the classification recognition result of the target in the radar multimode data of the target to be identified; The trained multi-mode radar data target fusion recognition model is obtained by training the initial multi-mode radar data target fusion recognition model using data of preset categories as a training data set; The preset categories of data include radar broadband high-resolution range image echoes, narrowband modulation spectrum echoes and target track sequences.

2. The intelligent target fusion recognition method based on radar multimodal data according to claim 1 is characterized in that: Before training the initial multi-mode radar data target fusion recognition model, it also includes: Construct a multi-mode radar data target fusion recognition model; among them, The multi-mode radar data target fusion recognition model includes multiple single-mode feature extraction modules, a multi-mode feature learning subspace constraint module, a similarity feature compression module, a feature reconstruction module and an attention feature fusion module; the single-mode feature extraction module is used to extract single-mode features, the multi-mode feature learning subspace constraint module includes an inter-modal similarity feature encoder and an intrinsic feature encoder, the inter-modal similarity feature encoder is used to encode all the single-mode features to obtain multiple inter-modal similarity features, the intrinsic feature encoder is used to encode the corresponding single-mode features to obtain intrinsic features, the similarity feature compression module is used to compress the multiple inter-modal similarity features to obtain multi-modal similarity features, the feature reconstruction module includes multiple decoders, which are used to reconstruct the corresponding single-mode features according to the multi-modal similarity features and the corresponding intrinsic features to obtain the loss of reconstructing each single-mode feature, the attention feature fusion module includes multiple attention fusion layers and classification layers, the attention fusion layer is used to fuse the multi-modal similarity features and the intrinsic features, and the classification layer is used to classify the fused features and output the recognition probability.

3. The intelligent target fusion recognition method based on radar multimodal data according to claim 2 is characterized in that: The preset category data is used as the training data set to train the initial multi-mode radar data target fusion recognition model, including: Acquire a training data set, wherein the training data includes a plurality of training samples, each of which includes a radar broadband high-resolution range image echo, a narrowband modulation spectrum echo, and a target track sequence; Labeling the training samples in the training data set to obtain true labels of the training samples; The training data set and the real labels of the training samples in the training data set are input into the initial multi-mode radar data target fusion recognition model for iterative training until a preset condition is met, thereby obtaining the trained multi-mode radar data target fusion recognition model.

4. The intelligent target fusion recognition method based on radar multimodal data according to claim 3 is characterized in that: Inputting the training data set and the true labels of the training samples in the training data set into the initial multi-mode radar data target fusion recognition model for iterative training, including: For the In the iterative training process, the training data set is input into the multi-mode radar data target fusion recognition model, and feature extraction is performed by the single-mode feature extraction module to obtain radar broadband high-resolution range image echo features, narrow-band modulation spectrum echo features and target track sequence features respectively; Input the radar broadband high-resolution range image echo feature, the narrowband modulation spectrum echo feature and the target track sequence feature into the multi-mode feature learning subspace constraint module for processing, and obtain the radar broadband high-resolution range image echo inter-modal similarity feature, the narrowband modulation spectrum echo inter-modal similarity feature, the target track sequence inter-modal similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrowband modulation spectrum echo intrinsic feature and the target track sequence intrinsic feature respectively; calculate the consistency loss according to the radar broadband high-resolution range image echo inter-modal similarity feature, the narrowband modulation spectrum echo inter-modal similarity feature and the target track sequence inter-modal similarity feature, and calculate the difference loss according to the radar broadband high-resolution range image echo inter-modal similarity feature, the narrowband modulation spectrum echo inter-modal similarity feature, the target track sequence inter-modal similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrowband modulation spectrum echo intrinsic feature and the target track sequence intrinsic feature; Inputting the radar broadband high-resolution range image echo inter-modal similarity feature, the narrowband modulation spectrum echo inter-modal similarity feature and the target track sequence inter-modal similarity feature into the similarity feature compression module for feature compression to obtain a multi-mode similarity feature; Input the multi-mode similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrow-band modulation spectrum echo intrinsic feature, the target track sequence intrinsic feature, the radar broadband high-resolution range image echo feature, the narrow-band modulation spectrum echo feature and the target track sequence feature into the feature reconstruction module for processing, and calculate the reconstruction loss; The multi-mode similarity features, the radar broadband high-resolution range image echo intrinsic features, the narrow-band modulation spectrum echo intrinsic features, the target track sequence intrinsic features and the true labels of the training samples in the training data set are input into the attention feature fusion module for processing to obtain the classification results, and the classification loss is calculated.

5. The intelligent target fusion recognition method based on radar multimodal data according to claim 4 is characterized in that: The preset condition includes that the sum of the consistency loss, the difference loss, the reconstruction loss and the classification loss is minimized during a finite number of iterations of training.

6. The intelligent target fusion recognition method based on radar multimodal data according to claim 4 is characterized in that: The acquisition process of the radar broadband high-resolution range image echo feature, the narrowband modulation spectrum echo feature and the target track sequence feature includes: The training data set is input into a radar broadband high-resolution range image echo feature extraction module, normalized and aligned in sequence, and then feature extracted to obtain radar broadband high-resolution range image echo features; The training data set is input into a narrowband modulation spectrum echo feature extraction module, and clutter removal, main body compensation and normalization are performed in sequence, and then feature extraction is performed to obtain narrowband modulation spectrum echo features; The training data set is input into the target track sequence feature extraction module, and format reorganization and normalization are performed in sequence, and then feature extraction is performed to obtain the target track sequence features; Among them, the single-mode feature extraction module includes the radar broadband high-resolution range image echo feature extraction module, the narrow-band modulation spectrum echo feature extraction module and the target track sequence feature extraction module.

7. The intelligent target fusion recognition method based on radar multimodal data according to claim 4 is characterized in that: Inputting the radar broadband high-resolution range image echo feature, the narrowband modulation spectrum echo feature and the target track sequence feature into the multi-mode feature learning subspace constraint module for processing, including: Inputting the radar broadband high-resolution range image echo feature, the narrowband modulation spectrum echo feature and the target track sequence feature into the inter-modal similarity feature encoder for processing, to obtain the radar broadband high-resolution range image echo inter-modal similarity feature, the narrowband modulation spectrum echo inter-modal similarity feature and the target track sequence inter-modal similarity feature; Inputting the radar broadband high-resolution range image echo feature into a radar broadband high-resolution range image echo intrinsic feature encoder for processing to obtain the radar broadband high-resolution range image echo intrinsic feature; Inputting the narrowband modulation spectrum echo feature into a narrowband modulation spectrum echo intrinsic feature encoder for processing to obtain the narrowband modulation spectrum echo intrinsic feature; Inputting the target track sequence features into a target track sequence intrinsic feature encoder for processing to obtain the target track sequence intrinsic features; Wherein, the single-mode feature encoder includes the radar broadband high-resolution range image echo intrinsic feature encoder, the narrowband modulation spectrum echo intrinsic feature encoder and the target track sequence intrinsic feature encoder; The consistency loss is calculated based on the similarity features between the radar broadband high-resolution range image echo modes, the similarity features between the narrowband modulation spectrum echo modes, and the similarity features between the target track sequence modes, and the expression is: ； in, represents the consistency loss, represents the feature vector, Representing modality and The corresponding inter-modality similarity feature pairs, Represents the similarity feature between modalities, It represents the radar broadband high-resolution range image echo mode. represents the narrowband modulation spectrum echo mode, represents the target track sequence mode, express The central moment difference, Indicates the maximum order of the calculated center distance; The difference loss is calculated according to the similarity characteristics between the radar broadband high-resolution range image echo modes, the similarity characteristics between the narrowband modulation spectrum echo modes, the similarity characteristics between the target track sequence modes, the radar broadband high-resolution range image echo intrinsic characteristics, the narrowband modulation spectrum echo intrinsic characteristics and the target track sequence intrinsic characteristics, and its expression is: ； in, represents the difference loss, Indicates different modes, Representing modality The corresponding inter-modality similarity features, Representing modality The corresponding intrinsic features, represents the Frobenius norm, Representing modality The corresponding intrinsic features, Representing modality The corresponding intrinsic features, Indicates transpose.

8. The intelligent target fusion recognition method based on radar multimodal data according to claim 4 is characterized in that: Inputting the multi-mode similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrow-band modulation spectrum echo intrinsic feature, the target track sequence intrinsic feature, the radar broadband high-resolution range image echo feature, the narrow-band modulation spectrum echo feature and the target track sequence feature into the feature reconstruction module for processing, including: Inputting the multimode similarity feature and the radar broadband high-resolution range image echo intrinsic feature into a radar broadband high-resolution range image echo feature decoder to obtain a reconstructed radar broadband high-resolution range image echo feature; calculating the radar broadband high-resolution range image echo feature reconstruction loss according to the reconstructed radar broadband high-resolution range image echo feature and the radar broadband high-resolution range image echo feature; Inputting the multi-mode similarity feature and the narrowband modulation spectrum echo intrinsic feature into a narrowband modulation spectrum echo feature decoder to obtain a reconstructed narrowband modulation spectrum echo feature; calculating the narrowband modulation spectrum echo feature reconstruction loss according to the reconstructed narrowband modulation spectrum echo feature and the narrowband modulation spectrum echo feature; Inputting the multi-mode similarity feature and the target track sequence intrinsic feature into a target track sequence feature decoder to obtain a reconstructed target track sequence feature; calculating the target track sequence feature reconstruction loss according to the reconstructed target track sequence feature and the target track sequence feature; The decoder includes the radar broadband high-resolution range image echo feature decoder, the narrowband modulation spectrum echo feature decoder and the target track sequence feature decoder; Among them, the expression of the radar broadband high-resolution range image echo feature reconstruction loss, the narrowband modulation spectrum echo feature reconstruction loss or the target track sequence feature reconstruction loss is: ； in, represents the reconstruction loss, Indicates different modes, represents the mean square error, Represents the modality extracted by the feature extractor Features, Represents the reconstructed mode Features, It represents the radar broadband high-resolution range image echo mode. represents the narrowband modulation spectrum echo mode, represents the target track sequence mode, represents the Frobenius norm.

9. The intelligent target fusion recognition method based on radar multimodal data according to claim 4 is characterized in that: Inputting the multi-mode similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrowband modulation spectrum echo intrinsic feature, the target track sequence intrinsic feature and the real label of the training sample in the training data set into the attention feature fusion module for processing, including: The multi-mode similarity feature, the radar broadband high-resolution range image echo intrinsic feature, the narrowband modulation spectrum echo intrinsic feature and the target track sequence intrinsic feature are sequentially feature mapped and position encoded, and then input into the multi-layer attention fusion layer for feature fusion processing, and then input into the classification layer for classification to obtain the classification result; the classification loss is calculated according to the classification result and the true label of the training sample in the training data set, and its expression is: ； in, represents the classification loss, represents the total number of samples, represents the sample index, represents the true label, Represents the predicted label.

10. An intelligent target fusion recognition system based on radar multimodal data, used to implement the intelligent target fusion recognition method based on radar multimodal data as described in any one of claims 1 to 9, characterized in that: include: A data acquisition module, used to acquire radar multi-mode data of the target to be identified; A data processing module, used for inputting the radar multimode data of the target to be identified into a trained multimode radar data target fusion recognition model, extracting single-mode features in sequence to obtain a plurality of single-mode features respectively, and performing inter-modal similarity feature and intrinsic feature learning based on multi-modal feature learning subspace constraints according to the plurality of single-mode features, compressing the learned multiple inter-modal similarity features to obtain multi-modal similarity features, performing attention fusion of the learned multiple intrinsic features and the multi-modal similarity features, and outputting a recognition probability to obtain a classification recognition result of the target in the radar multimode data of the target to be identified; The trained multi-mode radar data target fusion recognition model is obtained by training the initial multi-mode radar data target fusion recognition model using data of preset categories as a training data set; The preset categories of data include radar broadband high-resolution range image echoes, narrowband modulation spectrum echoes and target track sequences.

Images