Space target single view ISAR three-dimensional reconstruction method based on MS-PCNet

Abstract

The invention discloses a space target single view ISAR three-dimensional reconstruction method based on MS-PCNET, and the method comprises the steps: carrying out the mask extraction of an ISAR image, and obtaining a mask image; performing depth feature extraction on the mask image to generate mask image features; obtaining point cloud features, global features and ISAR image features; splicing the point cloud features, the global features, the ISAR image features and the mask image features, and inputting the spliced features into a basic full-connection decoder for processing to obtain a point cloud model; an orthogonal projection matrix is obtained through a radar line-of-sight angle, a mapping relation between an ISAR image and a target point cloud is established by using the orthogonal projection matrix, and pixel points in the ISAR image are mapped to a point cloud model. According to the method, the noise influence is eliminated, the basic structure of the target is ensured, the feature extraction precision, robustness and three-dimensional structure perception capability are remarkably improved, and the three-dimensional reconstructed point cloud model is good in effect.

Description

Technical Field

[0001] The invention belongs to the technical field of signal processing and relates to a space target single-view ISAR three-dimensional reconstruction method based on MS-PCNet. Background Art

[0002] Inverse Synthetic Aperture Radar (ISAR), with its numerous advantages, including all-day, all-weather capability, long range, and high resolution, plays a crucial role in space situational awareness and aerial target surveillance. However, ISAR observations often target non-cooperative targets, making it difficult to accurately obtain their complete attitude information from ISAR images. Achieving three-dimensional reconstruction of ISAR images and successfully constructing effective point cloud models would greatly advance situational awareness and surveillance of non-cooperative targets. However, the reality is that ISAR images exhibit significant characteristics such as low signal-to-noise ratio and lack of depth information. These factors make it difficult for traditional three-dimensional reconstruction methods to reconstruct practically applicable three-dimensional point cloud models when used with ISAR images.

[0003] In the existing technology, single-view 3D reconstruction methods for ISAR images can be mainly divided into the following categories: 1) Methods based on motion compensation and imaging model inversion. Leveraging the target's micro-motion characteristics (such as rotation and swing) or multi-view observations from a platform, the target's motion trajectory or posture changes in 3D space are estimated from multiple ISAR images. Combined with the imaging geometry, a 3D point cloud or contour is inferred. 2) Methods based on sparse point cloud extraction and geometric modeling. Scattering centers are extracted from the ISAR image through energy peak detection, pixel clustering, and other methods to construct a sparse point cloud. The point cloud is then parameterized and fitted using prior geometric primitives (such as spheres, cylinders, and cuboids) to optimize the 3D structure. 3) Deep learning-based shape-to-shape learning methods. Using a data-driven approach, neural networks (CNN, Transformer, PointNet, etc.) are trained to directly learn the mapping relationship from ISAR images to 3D models (point clouds / voxels / grids).

[0004] However, methods based on motion compensation and imaging model inversion require high-precision motion estimation and imaging model inversion, placing high demands on image quality and observation conditions. Methods based on sparse point cloud extraction and geometric modeling rely heavily on prior knowledge for point cloud extraction and primitive selection, potentially limiting their versatility. Deep learning-based graph-to-shape learning methods require large amounts of labeled data for training, and the model's generalization capabilities may be limited. Summary of the Invention

[0005] The present invention aims to solve the technical problem that the current ISAR image quality is low and it is difficult to reconstruct an effective 3D point cloud model. The present invention provides a single-view ISAR 3D reconstruction method for space targets based on MS-PCNet. The technical solution adopted is:

[0006] A single-view ISAR 3D reconstruction method for space targets based on MS-PCNET, comprising the following steps:

[0007] S1. Perform mask extraction on the ISAR image to obtain a mask image;

[0008] S2. performing depth feature extraction on the mask image to generate mask image features;

[0009] S3, obtaining point cloud features, global features and ISAR image features;

[0010] S4, performing matrix concatenation on the point cloud features, the global features, the ISAR image features, and the mask image features, and inputting the concatenation into a basic fully connected decoder for processing to obtain a point cloud model;

[0011] S5. Obtain an orthogonal projection matrix according to the radar line of sight angle, establish a mapping relationship between the ISAR image and the target point cloud using the orthogonal projection matrix, and map the pixels in the ISAR image to the point cloud model.

[0012] In one embodiment of the present invention, in step S1, U 2 -Net network architecture performs mask extraction on the ISAR image, and the U 2 The -Net network architecture captures contextual information by nesting the U-Net structure within each block.

[0013] In one embodiment of the present invention, step S1 includes:

[0014] S11, electromagnetically simulating the ISAR image and the mask image corresponding to the ISAR image to obtain a simulated ISAR image and a simulated mask image;

[0015] S12, performing data enhancement on the simulated ISAR image to obtain an enhanced ISAR image;

[0016] S13, dividing the enhanced ISAR image and the simulated mask image into a training set and a test set, and inputting the training set into the U 2 -Net network architecture to obtain the trained U 2 -Net network architecture;

[0017] S14, input the test set into the trained U2 -Net network architecture for testing.

[0018] In one embodiment of the present invention, step S13 includes:

[0019] The U 2 The -Net network architecture uses the Adam optimizer during training and sets the initial learning rate to 1×10 -4 ;

[0020] In terms of loss function, cross entropy loss is selected as the optimization target, and the calculation formula is expressed as:

[0021]

[0022] In formula (1), N = H × W, which represents the total number of pixels in the image, H represents the height of the image, and W represents the height of the image; y i Indicates that the true label is the true mask image of the input. When y i =1, it means pixel i is the target area, y i = 0, it means pixel i is the background area, p i Represents the predicted probability, ranging from 0 to 1, and the closer it is to 1, the more likely it is the target area;

[0023] The data set uses the enhanced ISAR image and the simulated mask image, and is divided into a training set and a validation set in a ratio of 4:1.

[0024] In one embodiment of the present invention, in step S2, a CNN-Transformer network architecture is used to perform deep feature extraction on the mask image to generate mask image features.

[0025] In one embodiment of the present invention, the CNN-Transformer network architecture includes a first convolution block, a second convolution block, a third convolution block, an attention module and a fifth convolution block;

[0026] The first convolutional block contains 6 convolutional layers, the first two convolutional layers have 16 output channels, the next three convolutional layers have 32 output channels, and the last convolutional layer has 64 output channels. The stride is 2 for downsampling operation.

[0027] The second convolutional block contains three convolutional layers, all of which have 128 output channels and a stride of 2.

[0028] The third convolution block contains 3 convolution layers, the number of output channels is 256, the first convolution layer uses a 5*5 convolution kernel with a stride of 2;

[0029] The attention module introduces the Transformer structure for global feature modeling;

[0030] The fifth convolution block contains 4 convolution layers, the number of output channels is 512, and the first convolution layer uses a 5*5 convolution kernel with a step size of 2.

[0031] In one embodiment of the present invention, step S3 includes:

[0032] S31, randomly generating an initial random point cloud as the starting point of 3D reconstruction;

[0033] S32, multiplying the coordinate values ​​of the initial random point cloud by a predefined orthogonal projection matrix, and accurately mapping the transformed coordinates to corresponding positions on the multi-scale feature map through bilinear interpolation to obtain point cloud features;

[0034] S33, extracting global features from the mask image features using the AdaIN method to obtain global features;

[0035] S34. Using CNN-Transformer to extract features from the ISAR image to obtain ISAR image features.

[0036] In one embodiment of the present invention, step S33 includes:

[0037] The AdaIN method is expressed as:

[0038]

[0039] In formula (2), X i Represents the feature set of the mask feature map, σx i and μx i are the variance and mean corresponding to the feature map, σY i and μY i Represents the variance and mean corresponding to the features of the initial random point cloud, y j ∈Y i Represents the feature vector of the j-th point in the point cloud.

[0040] In one embodiment of the present invention, step S4 includes:

[0041] The basic fully connected decoder reconstructs a 3D point cloud from the point cloud features, the global features, the ISAR image features and the mask image features;

[0042] The basic fully connected decoder starts from the input layer with the input dimension of Dm→512Dm\to 512Dm→512, and uses the ReLU activation function to introduce nonlinearity;

[0043] A linear layer converts the 512-dimensional features to 256 dimensions and uses ReLU activation. Another linear layer reduces the dimension from 256 to 128 and uses ReLU activation.

[0044] The basic fully connected decoder ends with a linear layer that maps the 128-dimensional features to 3-dimensional outputs suitable for point cloud coordinates.

[0045] In one embodiment of the present invention, step S5 includes:

[0046] ISAR generates a two-dimensional image of the target by rotating the target relative to the radar. The two dimensions of the two-dimensional image are the distance dimension and the direction dimension.

[0047] The distance dimension is determined by the time delay of the radar signal and is parallel to the radar line of sight;

[0048] The directional dimension is determined by the Doppler shift caused by the rotation of the target and is perpendicular to the radar line of sight;

[0049] The projection matrix is ​​obtained based on the radar line of sight. Let u be the radar line of sight unit vector, the projection plane W is the subspace perpendicular to u, and the orthogonal basis of the projection plane consists of two unit vectors v1 and v2. The relationship between the three is expressed as: v1·u=0,v2·u=0,v1·v2=0;

[0050] Assume u=[u x ,u y ,u z ] T , take a vector e1 that is not parallel to u, and use Gram-Schmidt orthogonalization to get:

[0051]

[0052] Let the orthogonal projection matrix be P, The orthogonal projection formula is expressed as:

[0053] P=Q(Q T Q) -1 Q T , and because Q T Q = I2, so The specific form of P is expressed as:

[0054]

[0055] In formula (3), P represents the orthogonal projection matrix, v 1x ,v 1y ,v 1z ,v 2x ,v 2y ,v 2zRepresent the components of the unit vectors v1 and v2 on the x-axis, y-axis, and z-axis respectively.

[0056] Beneficial effects of the present invention:

[0057] The MS-PCNET-based single-view ISAR 3D reconstruction method for space targets of the present invention performs a mask extraction operation on the ISAR image before performing 3D reconstruction, thereby eliminating the influence of noise while ensuring the basic structure of the target; adopts the CNN-Transformer network to extract features from the mask image and the ISAR image, combining the advantages of local perception and global modeling, and can effectively mine the multi-level and multi-scale structure and texture information in the mask image and the ISAR image, significantly improving the accuracy, robustness and 3D structure perception ability of feature extraction; adds an ISAR image feature extraction branch, while ensuring the basic structure of the target through the mask image features, significantly improving the network's modeling ability of radar physical characteristics, the robustness of cross-modal fusion, and the richness and effectiveness of the overall feature expression. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flow chart of a method for single-view ISAR 3D reconstruction of a space target based on MS-PCNET provided by an embodiment of the present invention;

[0059] Figure 2 is an ISAR image and its mask image provided by an embodiment of the present invention;

[0060] Figure 3 is a schematic diagram of a satellite model used as a simulation object provided by an embodiment of the present invention;

[0061] Figure 4 The ISAR image and its real point cloud label map provided by the embodiment of the present invention;

[0062] Figure 5 This is a comparison chart of ICESat2 target reconstruction results provided by an embodiment of the present invention;

[0063] Figure 6 This is a comparison diagram of ACRimsat target reconstruction results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] Aiming at the problem that the current ISAR image quality is low and it is difficult to reconstruct an effective three-dimensional point cloud model, the present invention provides a single-view ISAR three-dimensional reconstruction method for space targets based on MS-PCNET.

[0066] Refer to the attached Figure 1The MS-PCNET-based single-view ISAR 3D reconstruction method for space targets includes the following steps:

[0067] S1. Perform mask extraction on the ISAR image to obtain a mask image;

[0068] S2, performing deep feature extraction on the mask image to generate mask image features;

[0069] S3, obtaining point cloud features, global features and ISAR image features;

[0070] S4: Matrix concatenation of point cloud features, global features, ISAR image features, and mask image features, and inputting them into the basic fully connected decoder for processing to obtain a point cloud model.

[0071] S5. Obtain an orthogonal projection matrix through the radar line of sight angle, use the orthogonal projection matrix to establish a mapping relationship between the ISAR image and the target point cloud, and map the pixels in the ISAR image to the point cloud model.

[0072] Before performing three-dimensional reconstruction of an ISAR image, a mask extraction operation is first performed on the ISAR image. Step S1 not only solves the impact of noise on three-dimensional reconstruction to a certain extent, but also ensures the integrity and consistency of the target structure in the image. Then, depth feature extraction is performed on the mask image. Since ISAR images only contain two-dimensional representation information and lack depth information, traditional optical three-dimensional reconstruction methods are difficult to directly extract depth information. The present invention derives an orthogonal projection matrix through the radar line of sight angle, and uses the orthogonal projection matrix to establish a mapping relationship between the ISAR image and the target point cloud, thereby mapping the pixel points in the ISAR image to a point cloud model in three-dimensional space. The present invention not only improves the reconstruction quality of ISAR images, but also provides new possibilities for the development and application of radar imaging technology.

[0073] The network architecture of the present invention is divided into three parts. The first part is the mask extraction network, which is used for mask extraction, and the obtained mask image is input into the second part; the second part is the CNN-Transformer convolutional network, which is used for feature extraction operations, and the obtained features are spliced ​​and input into the third part; the third part is the basic fully connected decoder, which is used to restore the three-dimensional point cloud model of the target.

[0074] In an optional embodiment of the present invention, step S1 includes:

[0075] S11, obtaining an electromagnetic simulated ISAR image and a mask image corresponding to the ISAR image, and obtaining a simulated ISAR image and a simulated mask image;

[0076] S12, performing data enhancement on the simulated ISAR image to obtain an enhanced ISAR image;

[0077] S13, divide the enhanced ISAR image and the simulated mask image into a training set and a test set, and input the training set into U 2 -Net network architecture to obtain the trained U 2 -Net network architecture;

[0078] S14. Input the test set into the trained U 2 -Net network architecture for testing.

[0079] Test results refer to the attached Figure 2 Attached Figure 2 In the figure, (a) represents the input ICESat2 target ISAR image, (b) represents the mask effect diagram of the ICESat2 target, (c) represents the ACRimsat target ISAR image, and (d) represents the mask effect diagram of the ACRimsat target.

[0080] By the attached Figure 2 It can be seen that the present invention adopts U 2 -Net network for mask extraction can solve the problem of low signal-to-noise ratio of ISAR images to a certain extent, and provide strong support for subsequent three-dimensional reconstruction work.

[0081] The present invention adopts U 2 -Net network architecture is used to extract masks from ISAR images. The network is a deeply nested U-shaped structure, the core of which is a novel residual U-shaped module (RSU), which can extract multi-scale features without reducing the resolution of the feature map. Specifically, U 2 -Net achieves richer context information capture and a deeper network structure by nesting the U-Net structure within each block.

[0082] The present invention uses FEKO software electromagnetic simulation ISAR images and the mask images corresponding to the ISAR images. The dataset used consists of simulated ISAR images and their corresponding simulated mask images. These images are generated by electromagnetic simulation technology and can effectively simulate the ISAR image features in real environments, providing high-quality training samples for mask extraction. During the training process, U 2 After 100 batches of iterative training, the -Net network can well meet the demand for mask images. The present invention performs well in the mask extraction task and can accurately extract the target area. The effect is as shown in the attached figure. Figure 2 shown.

[0083] Considering the low signal-to-noise ratio of ISAR images, U 2The -Net network architecture uses the Adam optimizer during training, improves the stability of training through the adaptive learning rate adjustment mechanism, and sets the initial learning rate to 1×10 -4 , to avoid skipping the global optimal solution during training.

[0084] In terms of loss function, given the wide application of cross entropy loss function in mask extraction tasks and its accuracy in pixel-level classification evaluation, cross entropy loss is selected as the optimization target, and the calculation formula is expressed as:

[0085]

[0086] In formula (1), N = H × W, which represents the total number of pixels in the image, H represents the height of the image, and W represents the height of the image; y i Indicates that the true label is the true mask image of the input. When y i =1, it means pixel i is the target area, y i = 0, it means pixel i is the background area, p i Represents the predicted probability, ranging from 0 to 1, and the closer it is to 1, the more likely it is the target area.

[0087] The data set is divided into training set and validation set according to the ratio of 4:1 using enhanced ISAR images and simulated mask images. 2 The purpose of the -Net network architecture is to train a model that can quickly process ISAR images to obtain the corresponding mask images, and input the mask images and ISAR images into the subsequent 3D reconstruction network.

[0088] The present invention performs a mask extraction operation before 3D reconstruction, which greatly reduces the impact of ISAR image noise on subsequent 3D reconstruction operations and also ensures the integrity of the target structure in the ISAR image. Experimental verification shows that this scheme greatly improves the effect of 3D reconstruction and can provide good support for subsequent 3D reconstruction operations.

[0089] In step S2 of the present invention, the CNN-Transformer network architecture is used to extract deep features of the mask image and generate multi-scale mask image features to fully capture the detailed information and structural features of the target. The CNN-Transformer network architecture consists of a convolutional coding module and a MobileViTAttention module, wherein the conv layer group adopts a cascaded convolution design, and extracts layer by layer through multiple spatial downsampling, from shallow texture to deep semantics, abstracting layer by layer, and compressing spatial information through multi-level downsampling of multiple conv layer groups to improve the receptive field. The conv operation of each layer can be condensed into a formula, expressed as: i=σ(W i *s i-1 +b i ),i=1,…,k, where W i and b i Respectively represent the convolution kernel and bias of the i-th layer, σ represents the activation function. The present invention adopts the ReLU function as the activation function, * represents the two-dimensional convolution operation, s i-1 is the input of layer i, s i is the output of the i-th layer.

[0090] Preferably, the CNN-Transformer network architecture includes the first convolution block, the second convolution block, the third convolution block, the attention module and the fifth convolution block. The first convolution block contains 6 convolution layers, the output channels of the first two convolution layers are 16, the output channels of the next three convolution layers are 32, and the output channels of the last convolution layer are 64, with a stride of 2 for downsampling operations. The second convolution block contains three convolution layers, the output channels of all layers are 128, and the stride is also 2. The third convolution block contains 3 convolution layers, the output channels of all layers are 256, the first convolution layer uses a 5*5 convolution kernel, and the stride is 2. The attention module introduces the Transformer structure for global feature modeling. The fifth convolution block contains 4 convolution layers, the output channels are 512, and the first convolution layer uses a 5*5 convolution kernel with a stride of 2.

[0091] After obtaining the feature map of the mask image, the AdaIN method is used to perform global style migration. The AdaIN method is expressed as:

[0092]

[0093] In formula (2), X i Represents the feature set of the mask feature map, σx i and μx i are the variance and mean corresponding to the feature map, σY i and μY i Represents the variance and mean corresponding to the features of the initial random point cloud, y j ∈Y i Represents the feature vector of the j-th point in the point cloud.

[0094] In an optional embodiment of the present invention, step S3 includes:

[0095] S31, randomly generating an initial random point cloud as the starting point of 3D reconstruction;

[0096] S32, multiplying the coordinate values ​​of the initial random point cloud by a predefined orthogonal projection matrix, and accurately mapping the transformed coordinates to corresponding positions on the multi-scale feature map through bilinear interpolation to obtain point features;

[0097] S33, extracting global features from the mask image features using the AdaIN method to obtain global features;

[0098] S34. Use CNN-Transformer to extract features from ISAR images and obtain ISAR image features.

[0099] The present invention adds a branch for ISAR image feature extraction, which not only ensures the basic structure of the target through mask image features, but also significantly improves the network's modeling ability of radar physical characteristics, the robustness of cross-modal fusion, and the richness and effectiveness of overall feature expression.

[0100] In an optional embodiment of the present invention, step S4 includes:

[0101] The basic fully connected decoder reconstructs 3D point clouds from point cloud features, global features, ISAR image features, and mask image features, and gradually upsamples and optimizes feature representation through a multi-layer structure.

[0102] The basic fully connected decoder starts from the input layer with the input dimension of Dm→512Dm\to 512Dm→512, and then uses the ReLU activation function to introduce nonlinearity.

[0103] Next is a linear layer that converts the 512-dimensional features to 256 dimensions, using ReLU activation, followed by another linear layer that reduces the dimension from 256 to 128, also using ReLU activation. Finally, the basic fully connected decoder ends with a linear layer that maps the 128-dimensional features to 3-dimensional outputs suitable for point cloud coordinates.

[0104] This hierarchical approach combines linear transformation and ReLU activation to ensure a smooth transition from the high-dimensional feature space to the final 3D coordinate space, effectively reconstructing a point cloud with enhanced spatial details to obtain a point cloud model.

[0105] In an optional embodiment of the present invention, step S5 includes:

[0106] ISAR generates a two-dimensional image of the target by rotating the target relative to the radar. The two dimensions of the two-dimensional image are the distance dimension and the direction dimension.

[0107] The range dimension is determined by the time delay of the radar signal and is parallel to the radar line of sight (LOS);

[0108] The directional dimension is determined by the Doppler shift caused by the target's rotation and is perpendicular to the radar's line of sight (LOS);

[0109] The projection matrix is ​​obtained based on the radar line of sight. Let u be the radar line of sight unit vector, the projection plane W is the subspace perpendicular to u, and the orthogonal basis of the projection plane consists of two unit vectors v1 and v2. The relationship between the three is expressed as: v1·u=0,v2·u=0,v1·v2=0;

[0110] Assume u=[u x ,u y ,u z ] T , take a vector e1 that is not parallel to u, and use Gram-Schmidt orthogonalization to get:

[0111]

[0112] Let the orthogonal projection matrix be P, The orthogonal projection formula is expressed as:

[0113] P=Q(Q T Q) -1 Q T , and because Q T Q = I2, so The specific form of P is expressed as:

[0114]

[0115] In formula (3), P represents the orthogonal projection matrix, v 1x ,v 1y ,v 1z ,v 2x ,v 2y ,v 2z Represent the components of the unit vectors v1 and v2 on the x-axis, y-axis, and z-axis respectively.

[0116] The present invention uses ISAR images and orthogonal projection matrices as input training data, and uses chamfer distance as a loss function to calculate the similarity between the predicted point cloud and the real point cloud. Through continuous training and optimization, a model with the highest similarity is obtained. The obtained model can predict the point cloud model based on the input ISAR image and the functional relationship obtained through continuous training. This functional relationship is the mapping relationship learned by the model during training, that is, the mapping relationship between the ISAR image and the target point cloud. The ISAR image and orthogonal projection matrix are input into the CNN-Transformer network for training. The network learns the mapping relationship through continuous optimization iterations, thereby mapping the pixels in the ISAR image to the point cloud model.

[0117] Save the orthogonal projection matrix data, divide the data set, and input it into the network for training. The data ratio of the training set to the test set is 4:1. In this example, 300 cycles of training were performed to obtain experimental results. At the beginning of training, the learning rate was set to 0.0001, the cosine annealing function was used as the learning rate scheduler, and the Adam optimizer was used to optimize the regression target.

[0118] Finally, the model's effectiveness was tested and verified. The trained model was tested by inputting an ISAR image and observing the resulting point cloud. An untrained ISAR image was fed into the model trained for 300 cycles, resulting in a predicted point cloud model. This point cloud model served as the final result. The chamfer distance was then evaluated based on the point cloud model and the actual point cloud. A smaller value was preferred.

[0119] The evaluation indicators of the reconstruction effects of different algorithms are shown in Table 1.

[0120] Table 1

[0121]

[0122] As can be seen from Table 1, the lower the chamfer distance, the better the effect, and the higher the intersection-union ratio, the better the effect.

[0123] The present invention selects 34 different types of satellite models as simulation objects. Typical satellite model examples are shown in the attached figure. Figure 3 As shown. Figure 3 In the figure, the first row shows optical images of satellites such as ACE, TDRS, Jason-1, Rosetta and ICESat-2, the second row shows their corresponding 3D geometric models, and the third row shows ISAR images generated by electromagnetic simulation.

[0124] ISAR images and their real point cloud labels are attached Figure 4 , attached Figure 4 In the figure, (a) represents the input ICESat2 target ISAR image, (b) represents the real point cloud label of the ICESat2 target, (c) represents the ACRimsat target ISAR image, and (d) represents the real point cloud label of the ACRimsat target.

[0125] Comparison of ICESat2 target reconstruction results with reference to the attached Figure 5 , attached Figure 5In the figure, (a) shows the result of 3D reconstruction using one ISAR image by the energy accumulation 3D reconstruction algorithm based on ISAR images, (b) shows the result of 3D reconstruction using 20 ISAR images by the energy accumulation 3D reconstruction algorithm based on ISAR images, (c) shows the reconstruction result of Mesh R-CNN, and (d) shows the reconstruction result of the proposed method MS-PCNet.

[0126] Comparison of ACRimsat target reconstruction results with reference to the attached Figure 6 , attached Figure 6 In the figure, (a) shows the result of 3D reconstruction using one ISAR image by the energy accumulation 3D reconstruction algorithm based on ISAR images, (b) shows the result of 3D reconstruction using 20 ISAR images by the energy accumulation 3D reconstruction algorithm based on ISAR images, (c) shows the reconstruction result of Mesh R-CNN, and (d) shows the reconstruction result of the proposed method MS-PCNet.

[0127] By the attached Figure 4 , Attachment Figure 5 and attached Figure 6 It can be seen that the single-view ISAR 3D reconstruction method of space targets based on MS-PCNET in the present invention is superior to the energy accumulation 3D reconstruction algorithm based on ISAR images (using 1 ISAR image) and the Mesh R-CNN algorithm in terms of the structural integrity of point cloud reconstruction and the density of point clouds, and the reconstruction effect is similar to that of the energy accumulation 3D reconstruction algorithm based on ISAR images (using 20 ISAR images). However, the timeliness of the single-view ISAR 3D reconstruction method of space targets based on MS-PCNET in the present invention is significantly better than that of this method.

[0128] Compared to methods based on motion compensation and imaging model inversion, the present invention does not rely on high-quality ISAR images or observation conditions. Furthermore, the present invention has a simple implementation structure. Experimental verification shows that the three-dimensional point cloud model reconstructed by the present invention is effective and can effectively support processing such as situational awareness. Compared to other deep learning-based methods, the present invention effectively solves the problem of ISAR images not containing depth information, providing excellent support for the model to obtain depth information. Furthermore, the present invention has a simple network structure, strong network interpretability, and requires less data for training. Experimental verification also shows that the point cloud effect obtained by the present invention is also optimized.

[0129] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A single-view ISAR 3D reconstruction method for space targets based on MS-PCNet, characterized by: Including steps: S1. Perform mask extraction on the ISAR image to obtain a mask image; S2. performing depth feature extraction on the mask image to generate mask image features; S3, obtaining point cloud features, global features and ISAR image features; S4, splicing the point cloud features, the global features, the ISAR image features, and the mask image features, and inputting the concatenation into a basic fully connected decoder for processing to obtain a point cloud model; S5. Obtain an orthogonal projection matrix according to the radar line of sight angle, establish a mapping relationship between the ISAR image and the target point cloud using the orthogonal projection matrix, and map the pixels in the ISAR image to the point cloud model.

2. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 1, characterized in that: In step S1, U 2 -Net network architecture performs mask extraction on the ISAR image, and the U 2 The -Net network architecture captures contextual information by nesting the U-Net structure within each block.

3. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 2, characterized in that: Step S1 includes: S11, electromagnetically simulating the ISAR image and the mask image corresponding to the ISAR image to obtain a simulated ISAR image and a simulated mask image; S12, performing data enhancement on the simulated ISAR image to obtain an enhanced ISAR image; S13, dividing the enhanced ISAR image and the simulated mask image into a training set and a test set, and inputting the training set into the U 2 -Net network architecture to obtain the trained U 2 -Net network architecture; S14, input the test set into the trained U 2 -Net network architecture for testing.

4. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 3, characterized in that: Step S13 includes: The U 2 The -Net network architecture uses the Adam optimizer during training and sets the initial learning rate to 1×10 -4 ; In terms of loss function, cross entropy loss is selected as the optimization target, and the calculation formula is expressed as: In formula (1), N = H × W, which represents the total number of pixels in the image, H represents the height of the image, and W represents the height of the image; y i Indicates that the true label is the true mask image of the input. When y i =1, it means pixel i is the target area, y i = 0, it means pixel i is the background area, p i Represents the predicted probability, ranging from 0 to 1, and the closer it is to 1, the more likely it is the target area; The data set uses the enhanced ISAR image and the simulated mask image, and is divided into a training set and a validation set in a ratio of 4:

1.

5. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 1, characterized in that: In step S2, a CNN-Transformer network architecture is used to perform deep feature extraction on the mask image to generate mask image features.

6. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 5, characterized in that: The CNN-Transformer network architecture includes the first convolution block, the second convolution block, the third convolution block, the attention module and the fifth convolution block; The first convolutional block contains 6 convolutional layers, the first two convolutional layers have 16 output channels, the next three convolutional layers have 32 output channels, and the last convolutional layer has 64 output channels. The stride is 2 for downsampling operation. The second convolutional block contains three convolutional layers, all of which have 128 output channels and a stride of 2. The third convolution block contains 3 convolution layers, the number of output channels is 256, the first convolution layer uses a 5*5 convolution kernel with a stride of 2; The attention module introduces the Transformer structure for global feature modeling; The fifth convolution block contains 4 convolution layers, the number of output channels is 512, and the first convolution layer uses a 5*5 convolution kernel with a step size of 2.

7. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 1, characterized in that: Step S3 includes: S31, randomly generating an initial random point cloud as the starting point of 3D reconstruction; S32, multiplying the coordinate values ​​of the initial random point cloud by a predefined orthogonal projection matrix, and accurately mapping the transformed coordinates to corresponding positions on the multi-scale feature map through bilinear interpolation to obtain point cloud features; S33, extracting global features from the mask image features using the AdaIN method to obtain global features; S34. Using CNN-Transformer to extract features from the ISAR image to obtain ISAR image features.

8. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 1, characterized in that: Step S33 includes: The AdaIN method is expressed as: In formula (2), X i Represents the feature set of the mask feature map, σx i and μx i are the variance and mean corresponding to the feature map, σY i and μY i Represents the variance and mean corresponding to the features of the initial random point cloud, y j ∈Y i Represents the feature vector of the j-th point in the point cloud.

9. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 1, characterized in that: Step S4 includes: The basic fully connected decoder reconstructs a 3D point cloud from the point cloud features, the global features, the ISAR image features and the mask image features; The basic fully connected decoder starts from the input layer with the input dimension of Dm→512Dm\to 512Dm→512, and uses the ReLU activation function to introduce nonlinearity; A linear layer converts the 512-dimensional features to 256 dimensions and uses ReLU activation. Another linear layer reduces the dimension from 256 to 128 and uses ReLU activation. The basic fully connected decoder ends with a linear layer that maps the 128-dimensional features to 3-dimensional outputs suitable for point cloud coordinates.

10. The method for single-view ISAR 3D reconstruction of space targets based on MS-PCNet according to claim 8, characterized in that: Step S5 includes: ISAR generates a two-dimensional image of the target by rotating the target relative to the radar. The two dimensions of the two-dimensional image are the distance dimension and the direction dimension. The distance dimension is determined by the time delay of the radar signal and is parallel to the radar line of sight; The directional dimension is determined by the Doppler shift caused by the rotation of the target and is perpendicular to the radar line of sight; The projection matrix is ​​obtained based on the radar line of sight. Let u be the radar line of sight unit vector, the projection plane W is the subspace perpendicular to u, and the orthogonal basis of the projection plane consists of two unit vectors v1 and v2. The relationship between the three is expressed as: v1·u=0,v2·u=0,v1·v2=0; Assume u=[u x ,u y ,u z ] T , take a vector e1 that is not parallel to u, and use Gram-Schmidt orthogonalization to get: Let the orthogonal projection matrix be P, The orthogonal projection formula is expressed as: P=Q(Q T Q) -1 Q T , and because Q T Q = I2, so The specific form of P is expressed as: In formula (3), P represents the orthogonal projection matrix, v 1x ,v 1y ,v 1z ,v 2x ,v 2y ,v 2z Represent the components of the unit vectors v1 and v2 on the x-axis, y-axis, and z-axis respectively.

Images