Classification method and device based on microwave and lidar observation data correlation features

Abstract

The application relates to a classification method and device based on microwave and laser radar observation data association features. The method comprises the following steps: an association feature learning network is constructed according to a feature decomposition autoencoder structure of a P3S network and a twin neural network, and the association feature learning network is trained, so that the association feature learning network can extract association features of observation data acquired by multiple sensors for multiple targets, and target classification is performed based on the association features of the multiple observation data. The method can effectively improve the accuracy of target classification.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, and in particular to a classification method and apparatus based on the correlation characteristics of microwave and lidar observation data. Background Technology

[0002] Association analysis, a data analysis technique in the field of data mining, is often used to discover meaningful connections between large amounts of data. These meaningful connections are based on a relationship between the behavior that generated the data. Currently, association analysis of various forms of data has wide applications in cross-modal retrieval and target association, and many studies have also utilized the powerful learning capabilities of neural networks to mine and utilize internal relationships within data through data-driven methods.

[0003] For microwave and lidar, when the sensors detect the same scene target, there is a correlation between the observation data of the two sensors. This correlation can constrain the target judgment. If this prior constraint can be reasonably utilized, the joint processing of multi-sensor data can be better realized, and the detection performance of the system can be improved.

[0004] In existing technologies, the joint target detection problem using multi-sensor data from microwave and lidar largely relies on simple combinations of features from single or multiple sensors, with relatively little consideration given to the interrelationships between these features. However, these interrelationships are crucial for the joint processing of cross-spectral data. Therefore, for the same detection scenario, mining the correlation characteristics of data from one-dimensional range profile observations of microwave and lidar sensors is beneficial for precise target detection and classification. Thus, a target classification method based on the correlation features of multi-sensor data from microwave and lidar is proposed. Summary of the Invention

[0005] Therefore, it is necessary to provide a classification method and apparatus based on the correlation features of microwave and lidar observation data, which can classify targets based on the correlation features of multi-sensor detection data, in order to address the above-mentioned technical problems.

[0006] A classification method based on the correlation features of microwave and lidar observation data, the method comprising:

[0007] Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0008] A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0009] Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an association feature learning network is constructed. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0010] Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

[0011] In one embodiment, the microwave radar echo data and lidar echo data are one-dimensional range image echoes of the target.

[0012] In one embodiment, the preprocessing of the echo data in the echo dataset includes:

[0013] The microwave radar echo data and lidar echo data are given the same sequence length, and normalization processing is performed on the microwave radar echo data and lidar echo data for different targets.

[0014] In one embodiment, when constructing a twin dataset based on the preprocessed echo dataset:

[0015] When the microwave radar echo data and lidar echo data in the training sample meet the matching requirements, the training sample is a positive sample.

[0016] If the microwave radar echo data and lidar echo data in the training sample do not meet the matching requirements, then the training sample is a negative sample.

[0017] The matching requirement is that the microwave radar echo data and lidar echo data in the training samples are obtained by microwave radar and lidar detecting the same target at the same azimuth and elevation angles.

[0018] In one embodiment, the step of constructing the associated feature learning network based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network includes:

[0019] The associated feature learning network is constructed by embedding the feature decomposition autoencoder structure into the Siamese neural network. The associated feature learning network includes two feature decomposition autoencoder structures that process microwave radar echo data and lidar echo data respectively.

[0020] Each of the aforementioned feature decomposition autoencoder structures includes a shared feature encoder, a private feature encoder, and a decoder.

[0021] In one embodiment, when the twin dataset is input into the associated feature learning network:

[0022] In the training sample, a pair of microwave radar echo data and lidar echo data are decomposed into shared features and private features by the corresponding shared feature encoder and private feature encoder, respectively. The private features and the corresponding shared features are then added together and used as the input of the corresponding decoder to obtain the reconstructed features of the microwave radar echo data and lidar echo data.

[0023] In one embodiment, the process of inputting the twin dataset into the association feature learning network for training, and obtaining the trained association feature learning network further includes:

[0024] Construct the total loss function based on the reconstruction loss function, the orthogonal loss function, and the shared loss function;

[0025] The gradient of the total loss function is calculated, and the parameters of the shared feature encoder, private feature encoder and decoder in the associated feature learning network are corrected in reverse according to the calculation structure until convergence, thus obtaining the trained associated feature learning network.

[0026] A classification device based on the correlation features of microwave and lidar observation data, characterized in that the device comprises:

[0027] The dataset acquisition module is used to acquire the target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0028] The twin dataset construction module is used to construct a twin dataset based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0029] The associated feature learning network training module is used to construct an associated feature learning network based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network. The Siamese dataset is input into the associated feature learning network for training to obtain a trained associated feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the associated feature learning network. The obtained shared features are the associated features of the pair of microwave radar echo data and lidar echo data.

[0030] The target classification module is used to input positive samples from the training samples into the trained association feature learning network to extract association features, and classify the multiple targets according to the association features.

[0031] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0032] Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0033] A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0034] Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an association feature learning network is constructed. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0035] Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

[0036] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0037] Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0038] A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0039] Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an association feature learning network is constructed. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0040] Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

[0041] The aforementioned classification method and apparatus based on the correlation features of microwave radar and lidar observation data constructs a correlation feature learning network by using the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, and trains it to extract the correlation features of observation data acquired by multiple sensors for multiple targets, and classifies the targets based on the correlation features of multiple observation data, effectively improving the accuracy of target classification. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating a classification method based on the correlation features of microwave and lidar observation data in one embodiment.

[0043] Figure 2 A schematic diagram of a three-dimensional model of five types of aircraft targets used in a simulation.

[0044] Figure 3 This is a schematic diagram illustrating the construction of positive and negative samples in a twin dataset in one embodiment;

[0045] Figure 4 This is a schematic diagram of the SiameseP3SNet associated feature learning network structure in one embodiment;

[0046] Figure 5This is a schematic diagram of the feature decomposition autoencoder structure of SiameseP3SNet in one embodiment.

[0047] Figure 6 This is a schematic diagram comparing normalized one-dimensional distance image features and correlation features in one embodiment, wherein... Figure 6 (a) is the normalized one-dimensional distance image feature. Figure 6 (b) is a correlation feature;

[0048] Figure 7 A schematic diagram of the convergence curve for MINE-based mutual information estimation of associated features;

[0049] Figure 8 The visualization results of feature dimensionality reduction are shown in the figure. Figure 8 (a) is a visualization of the dimensionality reduction result of the original one-dimensional distance image features. Figure 8 (b) Visualization of the dimensionality reduction results of the associated features;

[0050] Figure 9 This is a structural block diagram of a classification device based on the correlation features of microwave and lidar observation data in one embodiment;

[0051] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] like Figure 1 As shown, a classification method based on the correlation features of microwave and lidar observation data is provided, including the following steps:

[0054] Step S100: Obtain the target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave radar and lidar detecting and slicing multiple targets.

[0055] Step S110: Construct a twin dataset based on the preprocessed target echo dataset. The twin dataset includes multiple training samples. Each training sample includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0056] Step S120: Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an associated feature learning network is constructed. The Siamese dataset is input into the associated feature learning network for training to obtain the trained associated feature learning network. Among them, the microwave radar echo data and lidar echo data in the training samples are decomposed into corresponding shared features by the shared feature encoder in the associated feature learning network. The obtained shared features are the associated features of the pair of microwave radar echo data and lidar echo data.

[0057] Step S130: Input the positive samples from the training samples into the trained association feature learning network to extract association features, and classify the multiple targets according to the association features.

[0058] In this embodiment, cross-spectral data from microwave and lidar are analyzed from the perspective of correlation between multi-sensor observation data. Specifically, a data-driven approach is used to learn the correlation features between microwave and lidar multi-sensor data to achieve joint target classification, increasing the interpretability of the extracted features. Furthermore, an improved Siamese neural network correlation feature learning model is provided. This model embeds the feature decomposition autoencoder structure of the P3SNet cross-modal retrieval model into the Siamese neural network to improve the effectiveness of correlation feature learning.

[0059] In step 100, the echo data is a collection of microwave radar echo data and lidar echo data obtained by microwave and lidar detection of different targets from multiple directions and at multiple elevation angles. Furthermore, this echo data is a one-dimensional range image echo of the target.

[0060] In verifying this method, radar echo data was obtained using RadBase electromagnetic simulation software at different advance frequencies. n},have And obtain the one-dimensional range image {x} of the target through IFFT. m},have

[0061] The laser echo data utilizes the target's .stl 3D model (the 3D model of the flying target used in the research is as follows). Figure 2 As shown, the full waveform one-dimensional range image echo of the target is obtained by using the surface element integration method.

[0062] In step S100, the preprocessing of the echo data in the echo dataset includes: taking the same sequence length for microwave radar echo data and lidar echo data, and normalizing the microwave radar echo data and lidar echo data for different targets.

[0063] Specifically, the radar and laser echo data were given the same sequence length L, which was chosen to be 256 in this study. Furthermore, to eliminate differences in the feature scales of one-dimensional range images from different sensors, the one-dimensional range image samples of different targets were normalized. Let the one-dimensional range image sequence of a certain sample be... Then the normalized distance image sequence {z n} can be represented as

[0064] In step S110, when constructing a twin dataset based on the preprocessed echo dataset: if the microwave radar echo data and lidar echo data in the training sample meet the matching requirements, then the training sample is a positive sample. If the microwave radar echo data and lidar echo data in the training sample do not meet the matching requirements, then the training sample is a negative sample. The matching requirements are: the microwave radar echo data and lidar echo data in the training sample are obtained by microwave radar and lidar from the same target at the same azimuth and elevation angles.

[0065] Specifically, Siamese datasets typically consist of matching positive samples and mismatched negative samples. Therefore, based on the original dataset, the construction method for Siamese neural network model data is as follows: Figure 3 As shown, when the azimuth, elevation, and target category of a pair of training samples are consistent, the microwave radar echo data and lidar echo data in that training sample match, making it a positive sample with a label L=1. Conversely, when the microwave radar echo data and lidar echo data in a pair of training samples are inconsistent, the two samples do not match, making them negative samples with a label L=0.

[0066] In step S120, by fully combining the advantages of P3SNet and Siamese neural network models, the SiameseP3SNet model (i.e., the correlation feature learning network) is proposed. This model embeds the feature decomposition autoencoder structure of the P3SNet network on the basis of the Siamese neural network.

[0067] Specifically, the associated feature learning network constructed based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network includes: embedding the feature decomposition autoencoder structure into the Siamese neural network to construct the associated feature learning network, which comprises two feature decomposition autoencoder structures for processing microwave radar echo data and lidar echo data, respectively. Each feature decomposition autoencoder structure includes a shared feature encoder, a private feature encoder, and a decoder, such as... Figure 4 As shown.

[0068] from Figure 4As can be seen above, the correlation feature learning network includes two branches, which process data from different modules, namely microwave radar echo data and lidar echo data. Each branch includes a shared feature encoder, a private feature encoder, and a decoder. Each branch is an embedded feature decomposition autoencoder structure.

[0069] like Figure 5 As shown, the autoencoder structure mainly consists of one-dimensional convolutional and deconvolutional layers, fully connected layers, ReLU (Rectified Linear Unit) loss function, and batch normalization (BN).

[0070] Furthermore, when the twin dataset is input into the associated feature learning network: a pair of microwave radar echo data and lidar echo data in the training samples are decomposed into shared features and private features by the corresponding shared feature encoder and private feature encoder, respectively. The private features and the corresponding shared features are then added together and used as the input of the corresponding decoder to obtain the reconstructed features of the microwave radar echo data and lidar echo data.

[0071] In this embodiment, the process of inputting the twin dataset into the association feature learning network for training, and obtaining the trained association feature learning network, further includes: constructing a total loss function based on the reconstruction loss function, the orthogonal loss function, and the shared loss function. Gradient calculation is performed on the total loss function, and the parameters of the shared feature encoder, the private feature encoder, and the decoder in the association feature learning network are corrected in reverse according to the calculation structure until convergence, thus obtaining the trained association feature learning network.

[0072] Specifically, a loss function is used for gradient optimization of parameter learning in the network parameter learning of the associated feature learning. The total loss function for training the SiameseP3SNet network can be expressed as Loss = αL recon +βL diff +L contrast , where α is L recon The weighting parameter, β is L diff The weight parameters. For the input radar and laser one-dimensional range image sample sets X and Y, the corresponding shared feature sample sets are H, respectively. s and G s The private feature sample set is H p and G p For the i-th pair of training samples input, let x i and y i The reconstructed features obtained after reconstruction using the SiameseP3SNet network are: and The obtained shared feature pairs are represented as and Private feature pairs are represented as and

[0073] The reconstruction loss function is used to constrain feature reconstruction. The more similar the reconstructed features are to the original input features, the higher the loss function becomes. Its calculation formula is expressed as:

[0074]

[0075] Where N represents the total number of samples, and the orthogonal loss function is used to constrain the independence of shared features and private features. The more independent they are, the better the loss function L becomes. diff The smaller the value of , the more its calculation formula is expressed as:

[0076]

[0077] Where T represents matrix transpose, F represents the Frobenius norm (F-norm), and the shared loss function is used to constrain the similarity between shared features of two modes (i.e., lidar echo data and microwave radar echo data). The more similar the shared features, the better the loss function L... sim The smaller the value, the more complex its calculation formula becomes:

[0078]

[0079] When learning network parameters, a training set is used for parameter gradient optimization, and a test set is used to store model parameters with smaller loss function values.

[0080] To gain a more intuitive understanding of the various loss functions mentioned above, you can refer to... Figure 4 .

[0081] In step S130, after obtaining the trained correlation feature learning network, correlation features can be extracted based on this network. All positive samples from the training samples (i.e., microwave radar and lidar one-dimensional range image sample pairs with consistent azimuth and elevation angles and the same target category) are input into the network. The trained correlation feature learning network model is used to extract shared features. The encoder structure output features of the shared features of the two input modalities are the correlation features to be studied, where x... output =Θ x (x input ), y output =Θ y (y input ), where Θ x and Θ y These are encoder network functions that share features in two modalities, x input and y inputThese are the input samples in two different modalities, x output and y output These are the shared feature outputs extracted from the input samples in the two modalities, respectively. For example... Figure 6 The image shown is a comparison between the learning results of the SiameseP3SNet model and the original one-dimensional distance image features.

[0082] Finally, the target is classified based on the fusion of the obtained associated features. Table 1 shows the classification accuracy of different features on one-dimensional distance image simulation datasets with different distance resolutions. It can be seen that the classification accuracy of associated features learned based on the SiameseP3SNet model proposed in this application is improved, indicating that multi-sensor target classification from the perspective of feature association analysis is effective.

[0083] Table 1 Comparison of SVM classification results

[0084]

[0085] This application also analyzes the correlation features to further demonstrate their importance in studying observational data obtained through multi-sensor target detection. This application estimates the mutual information of the correlation features based on the MINE neural network and visualizes the correlation features using t-SNE-based dimensionality reduction.

[0086] Specifically, the mutual information estimation of associated features is based on the MINE neural network. The calculation of the mutual information between high-dimensional variables X and Z is transformed into the calculation of the joint distribution P of the two variables. XZ Product with marginal distribution The calculation of the KL divergence, i.e. Furthermore, by the Donsker-Varadhan theorem... It can be seen that the KL divergence of distributions P and Q of two high-dimensional data variables can be transformed into a representation of the expectation of the two distributions by a function T. By choosing an optimal function T... option If the function on the right side of the equation is maximized, then this maximum value is the KL divergence value of distributions P and Q. The MINE network leverages the powerful data learning capabilities of neural networks to fit T. option This enables mutual information estimation between random variables of arbitrary dimensions and distributions. Network parameter optimization can be expressed as... The pseudocode for the MINE-based mutual information estimation algorithm for high-dimensional features is shown in Algorithm 1. The convergence curves for mutual information estimation of associated features are compared below. Figure 7 As shown.

[0087]

[0088] Specifically, t-SNE-based dimensionality reduction visualization of associated features is presented. t-SNE achieves non-linear dimensionality reduction of high-dimensional data by minimizing the difference between the similarity measure of paired sample points in the input and the similarity measure of paired points embedded in the corresponding low-dimensional data space, while effectively preserving the data's distribution patterns. The final visualization results of dimensionality reduction of matching samples for different features based on the t-SNE algorithm are shown below. Figure 8 As shown, where Figure 8 (a) is a visualization of the dimensionality reduction result of the original one-dimensional distance image features. Figure 8 (b) Visualization of the dimensionality reduction results of the associated features.

[0089] The aforementioned classification method based on the correlation features of microwave and lidar observation data proposes a correlation feature extraction model, SiameseP3Snet, under small sample conditions, based on existing Siamese neural network models and P3SNet cross-modal retrieval models. It learns the correlation features of microwave and lidar multi-sensor data through a data-driven approach to achieve joint target classification, thereby improving the accuracy of target classification. Furthermore, it fully explores the correlations between multi-sensor data using data mining methods. Based on the Siamese neural network, it embeds the feature decomposition autoencoder structure of the P3SNet cross-modal retrieval model, improving the learning effect of correlation features. Finally, it visually demonstrates the representational ability of correlation features on observation data by employing mutual information estimation and dimensionality reduction visualization methods.

[0090] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows; however, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed, and they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0091] In one embodiment, such as Figure 9 As shown, a classification device based on the correlation features of microwave and lidar observation data is provided, including: a dataset acquisition module 200, a twin dataset construction module 210, a correlation feature learning network training module 220, and a target classification module 230, wherein:

[0092] The dataset acquisition module 200 is used to acquire the target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0093] The twin dataset construction module 210 is used to construct a twin dataset based on the preprocessed echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0094] The association feature learning network training module 220 is used to construct an association feature learning network based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0095] The target classification module 230 is used to input the positive samples in the training samples into the trained association feature learning network to extract association features, and classify the multiple targets according to the association features.

[0096] Specific limitations regarding the classification device based on the correlation features of microwave and lidar observation data can be found in the limitations of the classification method based on the correlation features of microwave and lidar observation data mentioned above, and will not be repeated here. Each module in the aforementioned classification device based on the correlation features of microwave and lidar observation data can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0097] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a classification method based on the correlation features of microwave and lidar observation data. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0098] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0099] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0100] Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0101] A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0102] Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an association feature learning network is constructed. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0103] Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

[0104] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0105] Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets.

[0106] A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples.

[0107] Based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network, an association feature learning network is constructed. The Siamese dataset is input into the association feature learning network for training to obtain a trained association feature learning network. In the training samples, the microwave radar echo data and lidar echo data are decomposed into corresponding shared features by the shared feature encoder in the association feature learning network. The obtained shared features are the association features of the pair of microwave radar echo data and lidar echo data.

[0108] Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

[0109] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A classification method based on the correlation features of microwave and lidar observation data, characterized in that, The method for data processing and target classification from the perspective of data correlation includes: Acquire a target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets. A twin dataset is constructed based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples. A correlation feature learning network is constructed based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network. The Siamese dataset is input into the correlation feature learning network for training to obtain a trained correlation feature learning network. Specifically, the microwave radar echo data and lidar echo data in the training samples are decomposed into corresponding shared features by the shared feature encoder in the correlation feature learning network. The obtained shared features are the correlation features between the microwave radar echo data and the lidar echo data. The construction of the correlation feature learning network based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network includes: embedding the feature decomposition autoencoder structure into the Siamese neural network to construct the correlation feature learning network. The correlation feature learning network includes two feature decomposition autoencoder structures that process the microwave radar echo data and the lidar echo data respectively. Each feature decomposition autoencoder structure includes the shared feature encoder, the private feature encoder, and the decoder. Positive samples from the training samples are input into the trained association feature learning network to extract association features, and the multiple targets are classified according to the association features.

2. The classification method according to claim 1, characterized in that, The microwave radar echo data and lidar echo data are the one-dimensional range image echoes of the target.

3. The classification method according to claim 2, characterized in that, The preprocessing of the echo data in the echo dataset includes: The microwave radar echo data and lidar echo data are given the same sequence length, and normalization processing is performed on the microwave radar echo data and lidar echo data for different targets.

4. The classification method according to claim 1, characterized in that, When constructing a twin dataset based on the preprocessed echo dataset: When the microwave radar echo data and lidar echo data in the training sample meet the matching requirements, the training sample is a positive sample. If the microwave radar echo data and lidar echo data in the training sample do not meet the matching requirements, then the training sample is a negative sample. The matching requirement is that the microwave radar echo data and lidar echo data in the training samples are obtained by microwave radar and lidar detecting the same target at the same azimuth and elevation angles.

5. The classification method according to claim 4, characterized in that, When the twin dataset is input into the associated feature learning network: In the training sample, a pair of microwave radar echo data and lidar echo data are decomposed into shared features and private features by the corresponding shared feature encoder and private feature encoder, respectively. The private features and the corresponding shared features are then added together and used as the input of the corresponding decoder to obtain the reconstructed features of the microwave radar echo data and lidar echo data.

6. The classification method according to claim 5, characterized in that, The process of inputting the twin dataset into the association feature learning network for training further includes: Construct the total loss function based on the reconstruction loss function, the orthogonal loss function, and the shared loss function; The gradient of the total loss function is calculated, and the parameters of the shared feature encoder, private feature encoder and decoder in the associated feature learning network are corrected in reverse according to the calculation structure until convergence is obtained, thus obtaining the trained associated feature learning network.

7. A classification device based on the correlation characteristics of microwave and lidar observation data, characterized in that, The device includes: The dataset acquisition module is used to acquire the target echo dataset and preprocess the echo data in the target echo dataset. The echo data includes microwave radar echo data and lidar echo data obtained by microwave and lidar detection and slicing of multiple targets. The twin dataset construction module is used to construct a twin dataset based on the preprocessed target echo dataset. The twin dataset includes multiple training samples, each of which includes a pair of microwave radar echo data and lidar echo data. The training samples are divided into positive samples and negative samples. The associated feature learning network training module is used to construct an associated feature learning network based on the feature decomposition autoencoder structure of the P3S network and a Siamese neural network. The Siamese dataset is input into the associated feature learning network for training to obtain a trained associated feature learning network. Specifically, the microwave radar echo data and lidar echo data in the training samples are decomposed into corresponding shared features by the shared feature encoder in the associated feature learning network. The obtained shared features are the associated features of the pair of microwave radar echo data and lidar echo data. The construction of the associated feature learning network based on the feature decomposition autoencoder structure of the P3S network and the Siamese neural network includes embedding the feature decomposition autoencoder structure into the Siamese neural network to construct the associated feature learning network. The associated feature learning network includes two feature decomposition autoencoder structures that process the microwave radar echo data and lidar echo data respectively. Each feature decomposition autoencoder structure includes the shared feature encoder, a private feature encoder, and a decoder. The target classification module is used to input positive samples from the training samples into the trained association feature learning network to extract association features, and classify the multiple targets according to the association features.

Images