Method and system for three-dimensional reconstruction of space targets based on multi-view isar images
By using an energy accumulation method based on multi-view ISAR images and a differentiable rendering network, combined with a multi-resolution progressive optimization strategy, the problem of feature extraction difficulties in traditional ISAR image 3D reconstruction algorithms is solved, achieving higher accuracy and completeness in spatial target 3D reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2026-02-28
- Publication Date
- 2026-06-09
AI Technical Summary
Traditional 3D reconstruction algorithms for spatial targets based on ISAR images rely on feature extraction, which suffers from difficulties in feature extraction, significant noise impact, and insufficient reconstruction accuracy and reliability. This is especially true in cases of complex structures and multiple pose variations, leading to missing components and inaccurate reconstruction.
A method based on multi-view ISAR images is adopted. The SDF model is initialized by energy accumulation method. Combined with differentiable rendering network and multi-resolution progressive optimization strategy, a 3D reconstruction framework is directly constructed from the perspective of image rendering. Shading images are used for iterative optimization to generate a complete and accurate target 3D model.
It improves the anti-interference and generalization of 3D reconstruction, enhances the integrity and accuracy of the reconstruction model, and can effectively recover geometric shapes and local details under different targets and resolutions, thereby improving the accuracy and reliability of space target monitoring.
Smart Images
Figure CN122176170A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of 3D reconstruction technology, and in particular to a method and system for 3D reconstruction of spatial targets based on multi-view ISAR images. Background Technology
[0002] With the rapid development of aerospace technology, space safety issues in Earth's orbit are constantly emerging, posing numerous challenges and threats to on-orbit targets. For example, the increasing number of high-speed space debris is worsening the orbital environment, and even small debris can cause serious damage to on-orbit targets. Out-of-control satellites that have not yet been recovered may become potential collision hazards due to abnormal attitudes or energy depletion. Simultaneously, competition and coordination among spacecraft in utilizing orbital resources are becoming increasingly prominent, potentially leading to new safety hazards such as orbital conflicts. These factors exert multidimensional pressure on the normal operation of on-orbit space targets, making space situational awareness and space safety maintenance crucial issues.
[0003] To address these complexities, it is necessary to construct a monitoring network that integrates multi-source data to enhance the tracking, identification, and early warning capabilities of space targets. Inverse Synthetic Aperture Radar (ISAR), due to its all-weather, all-day, and long-range imaging capabilities, has become a crucial means of acquiring space target data. The ISAR images of on-orbit targets acquired through ISAR provide critical data support for the classification, identification, status assessment, and potential threat evaluation of space targets. Analysis of ISAR images not only yields basic physical parameters such as the target's geometry and size but also allows for the further inference of its structural characteristics, motion attitude, and even functional status, thus providing important evidence for determining whether the target exhibits abnormal behavior or potential risks.
[0004] Traditional 3D reconstruction algorithms for space targets based on ISAR images mostly rely on feature extraction, such as contour features and scattering point features, as exemplified by the 3D reconstruction method, apparatus, device, and storage medium based on neural radiation fields disclosed in invention publication number CN116310076A. However, the complex structure of space targets, diverse attitude changes, and potential noise and insufficient resolution during imaging often lead to difficulties or inaccuracies in feature extraction, thus affecting the accuracy and reliability of 3D reconstruction. Furthermore, due to factors such as target component occlusion, reconstructed models often suffer from missing components. Therefore, exploring 3D reconstruction methods that do not rely on precise feature extraction and possess stronger anti-interference capabilities and generalization ability has become an important research direction in the field of space target monitoring. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and system for three-dimensional reconstruction of spatial targets based on multi-view ISAR images. The method takes multi-view ISAR images and observation perspectives as input, eliminates the need for target feature extraction, constructs a three-dimensional reconstruction framework from the perspective of image rendering, and outputs a complete and accurate three-dimensional model of the target.
[0006] The objective of this invention can be achieved through the following technical solutions: A method for 3D reconstruction of spatial targets based on multi-view ISAR images includes: Acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points; calculate the projection position of the initial sampling points on the corresponding ISAR image based on the observation perspective of each ISAR image, and use the image pixel value of the projection position as the energy of the initial sampling point, thereby initializing the SDF model according to the energy accumulation method. Three-dimensional spatial sampling points are generated based on the observation viewpoint and imaging parameters of each ISAR image. The SDF value of each three-dimensional spatial sampling point is calculated using the SDF model. The target surface points are calculated based on the changes in the SDF values to generate a Shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image; The rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is iteratively optimized in reverse using a loss function to obtain the final SDF model, which serves as the result of the three-dimensional reconstruction of the space target.
[0007] Furthermore, the energy accumulation method includes: The energy of the initial sampling points on each ISAR image is accumulated, an adaptive threshold is calculated based on the maximum accumulated energy, and the initial sampling points with accumulated energy greater than the adaptive threshold are taken as target points, thereby constructing an initial point cloud; an initialized SDF model is constructed based on the initial point cloud.
[0008] Furthermore, the expression for the initial sampling point is: In the formula, It is the position index of a three-dimensional matrix. For resolution, It is the minimum boundary in three directions in three-dimensional space. For three-dimensional position The corresponding initial sampling point; The expression for the projection position of the initial sampling point on the corresponding ISAR image is: In the formula, Initial sampling point The row coordinates of the projection position on the m-th ISAR image. Initial sampling point The column coordinates of the projected position on the m-th ISAR image. and These are the height and width of the ISAR image, respectively. and These are the distance dimension resolution and the orientation dimension resolution, respectively. and These are the range dimension imaging axis and the azimuth dimension imaging axis, which are parameters belonging to the observation perspective; The formula for calculating the cumulative energy of the initial sampling points on each ISAR image is as follows: In the formula, Initial sampling point The cumulative energy on each ISAR image The total number of ISAR images. For ISAR images in Image pixel values at the location.
[0009] Furthermore, the process of generating the three-dimensional spatial sampling points includes: The sampling start point and sampling direction are calculated based on the observation angle and imaging parameters. The corresponding calculation expressions are as follows: In the formula, The sampling starting point, For the distance dimension imaging axis, As the azimuth imaging axis, The height of the imaging plane, The width of the imaging plane, For distance index, For direction index, For distance dimension resolution, For azimuth resolution, Sampling direction; From the starting point Begin, based on the given sampling range , ], along the sampling direction Sampling yields sampling points uniformly distributed in three-dimensional space. The corresponding calculation expression is: In the formula, Sampling direction The total number of samples.
[0010] Furthermore, the calculation of the target surface point based on the change in SDF value includes: Calculate the product of the SDF values of two adjacent 3D space sampling points. If the product is negative, select the point with the smaller SDF value from the two 3D space sampling points. Another point is The point is calculated using linear interpolation. and points New points between ; Calculate new points The SDF value, and compared with the preset threshold. Compare, if it is less than the threshold Then the new point As the target surface point; otherwise, the point ,point And new points Select two points with opposite signs and smaller SDF values as new points. and points Recalculate the new points until the new point The SDF value is less than the threshold Thus, it serves as the target surface point.
[0011] Furthermore, in the process of generating the shading image based on the target surface points, the expression for calculating the value of each pixel in the shading image is as follows: In the formula, For the first shading image pixel value, For pixel index, For the first The number of target surface points corresponding to each pixel value For the first The corresponding pixel One target surface point, To obtain the maximum value, The direction of the incident electromagnetic wave, For calculations based on the unit normal, the corresponding expression is: In the formula, For the SDF model matrix, For normalization, It is a three-dimensional index of the SDF model matrix.
[0012] Furthermore, the differentiable rendering network is a convolutional neural network; The loss function includes image rendering error loss, shape constraint loss, and regularization term.
[0013] Furthermore, a multi-resolution progressive optimization strategy is employed to perform reverse iterative optimization on the SDF model. This multi-resolution progressive optimization strategy includes: Multiple SDF model iterative optimization modules with different resolutions and iterative optimization parameters are set up. The iterative optimization parameters include the iteration termination condition. The SDF model iterative optimization modules are connected in sequence. The final SDF model output by the previous SDF model iterative optimization module is upsampled or downsampled to a specified resolution and used as the input of the next SDF model iterative optimization module. The processing steps of each SDF model iterative optimization module include: Three-dimensional spatial sampling points are generated based on the observation viewpoint and imaging parameters of each ISAR image. The SDF value of each three-dimensional spatial sampling point is calculated using the SDF model. The target surface points are calculated based on the changes in the SDF values to generate a Shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image; The rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is back-optimized using a loss function to obtain the final SDF model.
[0014] Furthermore, the method also includes converting the final SDF model into a voxel model and a mesh model for visualization, and calculating the crossover ratio (CROR) between the converted voxel model and the voxel model of the actual target as an evaluation index of model reconstruction accuracy, so as to quantitatively evaluate the reconstruction accuracy and determine whether the preset reconstruction accuracy requirements are met.
[0015] The present invention also provides a space target 3D reconstruction system for implementing the space target 3D reconstruction method based on multi-view ISAR images as described above, comprising: The model initialization module is used to acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points. Based on the observation perspective of each ISAR image, the projection position of the initial sampling point on the corresponding ISAR image is calculated, and the image pixel value of the projection position is used as the energy of the initial sampling point, thereby initializing the SDF model according to the energy accumulation method. The SDF model iterative optimization module is used to generate three-dimensional spatial sampling points based on the observation view and imaging parameters of each ISAR image. It calculates the SDF value of each three-dimensional spatial sampling point through the SDF model, calculates the target surface points based on the changes in the SDF value, and generates a shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image. The rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is back-optimized through a loss function. The multi-resolution progressive optimization module is used to set multiple SDF model iterative optimization modules with different resolutions and iterative optimization parameters connected in series. The output of the previous SDF model iterative optimization module is processed and used as the input of the next SDF model iterative optimization module. The 3D reconstruction model output module is used to visualize the final SDF model output by the multi-resolution progressive optimization module and calculate the reconstruction accuracy as the 3D reconstruction result of the spatial target.
[0016] Compared with the prior art, the present invention has the following advantages: (1) Strong anti-interference and generalization: In the initialization process of the SDF model, the cumulative energy is calculated based on the image pixel values of all input ISAR images at the initial sampling point projection position, thereby filtering target points and noise points and constructing an initial point cloud, which can effectively reduce the impact of noise on model initialization. Based on this, the SDF model iteratively optimized has strong anti-interference. Secondly, this invention uses Shading images instead of SDF models and observation perspectives as inputs to the differentiable rendering network, so that the differentiable rendering network only needs to learn the mapping between Shading images and ISAR images, and is not limited to a specific target model, thus it can be widely used for different targets.
[0017] (2) The present invention sets multiple SDF model resolutions and iterative optimization parameters, and adopts a multi-resolution progressive optimization strategy to iteratively optimize the SDF model. Multi-resolution progressive optimization helps to overcome the limitations of single-resolution reconstruction, restores the target geometry at low resolution, and reconstructs the local details of the target at high resolution. Furthermore, it does not limit the number and parameters of the SDF model iterative optimization modules. Therefore, the iterative parameter settings can be adjusted according to actual application needs, so that it has good performance in different target models, effectively improving the generalization ability of the invention in different application scenarios.
[0018] (3) Significantly improved accuracy and completeness of the reconstruction model: The rendered image output by the differentiable rendering network of the ISAR image in this invention maintains the same energy distribution as the multi-view ISAR image. The invisible parts in the multi-view ISAR image due to occlusion, noise, and other factors are also invisible in the rendered image. Therefore, when optimizing the SDF model according to the loss function, the adjustment of this part is reduced, and the integrity of the reconstruction model is guaranteed by this method. In addition, the multi-resolution progressive optimization strategy effectively enhances the optimization of model details and further improves the model reconstruction accuracy. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of model initialization provided in an embodiment of the present invention, wherein (a) is a point cloud model, (b) is a mesh model, and (c) is an SDF slice; Figure 2 This is a schematic diagram of three-dimensional point sampling provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of surface point retrieval provided in an embodiment of the present invention; Figure 4 This is a comparison image of an ISAR image and a Shading image provided in an embodiment of the present invention; Figure 5 This invention provides a three-dimensional model and simulation image of four typical spatial targets, wherein (a) is target I, (b) is target II, (c) is target III, and (d) is target IV; Figure 6 This is a U-Net rendering network structure diagram provided in an embodiment of the present invention; Figure 7 This is a comparison image of a simulated image and a rendered image provided in an embodiment of the present invention, wherein (a) is a simulated image and (b) is a rendered image; Figure 8 This is a schematic diagram of a multi-resolution progressive optimization process provided in an embodiment of the present invention, wherein the resolution and iteration in (a) are respectively , 0 th The resolution and iteration of (b) are respectively , 200 th The resolution and iteration of (c) are respectively 400 th The resolution and iteration of (d) are respectively 600 th The resolution and iteration of (e) are respectively 800 th The resolution and iteration of (f) are respectively , 100 th The resolution and iteration of (g) are respectively , 100 th ; Figure 9 This is a visualization result of a reconstruction model provided in an embodiment of the present invention; Figure 10 This is a flowchart illustrating a method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images provided in an embodiment of the present invention. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0023] Example 1 like Figure 10 As shown, this embodiment provides a method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images, including: S1: Acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points; calculate the projection position of the initial sampling points on the corresponding ISAR image based on the observation perspective of each ISAR image, and use the image pixel value of the projection position as the energy of the initial sampling point, thereby initializing the SDF model according to the energy accumulation method. S2: Generate three-dimensional spatial sampling points based on the observation view and imaging parameters of each ISAR image, calculate the SDF value of each three-dimensional spatial sampling point through the SDF model, calculate the target surface points based on the changes in the SDF value, and generate a Shading image. S3: Input the Shading image into the trained differentiable rendering network to obtain the rendered ISAR image; S4: Compare the rendered ISAR image with multi-view ISAR images, and perform reverse optimization of the SDF model using the loss function; S5: Employ a multi-resolution progressive optimization strategy to iteratively optimize the SDF model and obtain the final SDF model; S6: Visualize the final SDF model as the result of the three-dimensional reconstruction of the spatial target.
[0024] Specifically, step S1 includes: Uniform sampling is performed on a three-dimensional space with limited range. The projection position of the sampling point on the corresponding ISAR image is calculated based on the input observation view. The image pixel value at this position is used as the energy of the sampling point. The above operation is repeated to accumulate the energy of the sampling point on the input multi-view ISAR image. The dynamic threshold is calculated based on the maximum accumulated energy. Sampling points with accumulated energy greater than the dynamic threshold are used as part of the initialization target, and the point cloud is converted into an SDF model.
[0025] Compared with randomized initialization or sphere initialization strategies, the improved energy accumulation method for initializing the SDF model significantly reduces the running time and difficulty of subsequent SDF model iteration optimization.
[0026] The corresponding processing procedure includes the following sub-steps: S101: Uniform sampling in a three-dimensional space with limited boundaries is equivalent to calculating a series of points uniformly distributed in space, given a three-dimensional matrix and its corresponding boundary range and resolution, as shown in the following formula: (1) in It is an index of a three-dimensional matrix. For resolution, It represents the minimum boundary in three directions in actual three-dimensional space.
[0027] S102: Calculation In the The projection position on an ISAR image is given by the following formula: (2) (3) in, and These are the height and width of the image, respectively. and These are the distance dimension resolution and the orientation dimension resolution, respectively. These parameters belong to image parameters. and These are the range dimension imaging axis and the azimuth dimension imaging axis, respectively, which belong to the imaging observation viewpoint parameters.
[0028] S103: Calculate the cumulative energy value using the following formula: (4) In the formula, Initial sampling point The cumulative energy on each ISAR image The total number of ISAR images. For ISAR images in Image pixel values at the location.
[0029] S104: Will As an adaptive threshold, points with accumulated energy greater than the threshold are classified as target points, while those with less energy are classified as noise points, thus obtaining the initial point cloud. The point cloud model is converted into a mesh model using the boundary algorithm, and then the mesh model is converted into an SDF model using the mesh2SDF algorithm. The result is as follows. Figure 1 As shown.
[0030] Step S2 includes: using the ISAR imaging characteristics, generating three-dimensional spatial sampling points based on the observation angle and imaging parameters, calculating the SDF value of each sampling point through trilinear interpolation and SDF model, and calculating the target surface points interacting with the incident electromagnetic wave based on the changes in the SDF value. Each pixel position in the shading image corresponds to 0 or more target surface points. The inner product between the normal vector of these surface points and the incident electromagnetic wave is calculated, inverted, compared with 0, and the maximum value is taken. The sum is then used as the pixel value at that position.
[0031] The corresponding processing procedure includes the following sub-steps: S201: Calculate the sampling start point and sampling direction based on the imaging axis and imaging parameters, using the following formula: (5) (6) like Figure 2 As shown, from the starting point Begin, based on the given sampling range [ , Along the surface normal Sampling yields sampling points uniformly distributed in three-dimensional space. The corresponding calculation expression is: (7) in It represents the total number of samples in that direction.
[0032] S202: Surface point search. The SDF value of each sampling point is calculated using trilinear interpolation and the SDF model. Subsequently, the product of the SDF values of adjacent sampling points is calculated, such as... Figure 3 As shown, a negative product indicates that the two points are located on opposite sides of the target surface. Selecting from these... The smaller point is The other point is Calculated by linear interpolation The corresponding calculation expression is: (8) Calculate new points The SDF value is calculated and compared with a set threshold. If it is less than the threshold, the point is considered a surface point; otherwise, it is considered a surface point at two adjacent points and a new point. , , Choose two points with opposite signs and smaller SDF between them and recalculate. Until When the SDF value is less than the threshold, It is a target surface point that has been found.
[0033] S203: Calculate the pixel values of the shading image, using the following formula. (9) In the formula, For the first shading image pixel value, For pixel index, For the first The number of target surface points corresponding to each pixel value For the first The corresponding pixel One target surface point, To obtain the maximum value, The direction of the incident electromagnetic wave, For calculations based on the unit normal, the corresponding expression is: (10) In the formula, For the SDF model matrix, For normalization, It is a three-dimensional index of the SDF model matrix. Figure 4 A comparison of the calculated Shading image and the corresponding ISAR image is given.
[0034] Step S3 includes: using the Unet network, taking the Shading image as the network input and the ISAR image as the network output, and training the network accordingly to obtain a differentiable rendering network capable of rendering ISAR images.
[0035] Using shading images as network input instead of SDF models and observation perspectives helps reduce the difficulty of network training and enhance network generalization.
[0036] The corresponding processing procedure includes the following sub-steps: S301: Dataset preparation. For example... Figure 5 As shown, four typical space target models were constructed and multi-view ISAR images were simulated, and corresponding shading images were also generated. The ISAR image-shading image data pairs for targets I to III were divided into training and validation sets in an 8:2 ratio.
[0037] S302: Rendering network training. The U-Net network structure is as follows: Figure 6 As shown, the loss function is defined as: (11) in It's rendering an image. It is a simulated image. It is a pixel index. That is the total number of pixels.
[0038] The trained network was tested on target IV. Figure 7 These are the rendered images and corresponding simulation images output by the trained network. During subsequent iterative optimization, the weights of the trained rendering network remain frozen and do not participate in gradient updates or fine-tuning.
[0039] Step S4 includes: constructing a loss function based on the rendered ISAR image and the input ground truth to perform reverse optimization of the SDF model; comparing the rendered ISAR image output in step S3 with the input multi-view ISAR image; calculating structural similarity and pixel value differences; and adding constraint terms for the target shape and regularization terms for the SDF model constraints to improve iterative optimization performance.
[0040] The corresponding processing procedure includes the following sub-steps: S401: Construct the loss function. The loss function includes image rendering error loss. and Shape constraint loss and regularization terms and . Shape images are generated using multi-surface points extracted from an SDF model, and the image is evaluated using the Multi Scale Structural Similarity Index Measure (MS-SSIM) to compare it with other shapes. Similarity between them; make Each point in The normal vector at that point tends to the unit vector; This ensures the reconstruction model remains smooth, as defined below: (12) (13) (14) (15) (16) (17) in, Represents pixels The corresponding number A surface point, It is the number of surface points. This indicates rounding down. Point The normal vector at that point, It is the Laplace operator.
[0041] S402: Backward optimization. The NAdam optimizer is used to iteratively optimize based on pre-defined training parameters.
[0042] The object of backpropagation is the SDF model, which is an N*N*N matrix, represented as follows: That is, in equation (12) .
[0043] The constraints for the target shape are: The regularization term for the constraints on the SDF model is: and , are the last three loss terms in equation (12). The constraint term for the target shape... It is based on the SDF model ( The generated target shape image; in the regularization term, To make the normal vector of each point in the SDF model approach the unit vector, This keeps the reconstruction model smooth.
[0044] Step S5 includes: setting multiple sets of SDF model resolutions and iterative optimization parameters, taking steps S2-S4 as an SDF model iterative optimization module, and using the final SDF model output by the previous SDF model iterative optimization module as the input of the next SDF model iterative optimization module after being upsampled to the specified resolution.
[0045] Multi-resolution progressive optimization helps overcome the limitations of single-resolution reconstruction, restoring the target geometry at low resolution and reconstructing the target's local details at high resolution.
[0046] In this embodiment, the resolution includes , , The SDF model iterates 800, 100, and 100 times respectively at these three resolutions. The actual iterative optimization process is as follows: Figure 8 As shown.
[0047] Step S6 includes: SDF model, as an implicit representation, is not conducive to the analysis of model reconstruction. Converting SDF model into voxel and mesh model can intuitively show the geometric shape and detailed features of the reconstructed target. Then, the crossover ratio between the generated voxel model and the voxel model of the actual target is used as the evaluation index of model reconstruction accuracy to quantitatively evaluate the reconstruction accuracy.
[0048] The corresponding processing procedure includes the following sub-steps: S601: SDF Model Visualization. Utilizing the characteristic that the target surface has an SDF value of 0, the SDF model is converted into a voxel model and a mesh model. The visualization results are as follows: Figure 9 As shown.
[0049] S602: Calculate the reconstruction accuracy using the following formula: (18) in, Indicates intersection, Represents the union, It is a reconstructed voxel model. It is a voxel model of the actual target. In this embodiment, the reconstructed target IV... It is 0.7495.
[0050] Example 2 This embodiment provides a spatial target 3D reconstruction system that implements the spatial target 3D reconstruction method based on multi-view ISAR images as described in Embodiment 1, including: The model initialization module is used to acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points. Based on the observation perspective of each ISAR image, the projection position of the initial sampling point on the corresponding ISAR image is calculated, and the image pixel value of the projection position is used as the energy of the initial sampling point, thereby initializing the SDF model according to the energy accumulation method. The SDF model iterative optimization module is used to generate three-dimensional spatial sampling points based on the observation view and imaging parameters of each ISAR image. It calculates the SDF value of each three-dimensional spatial sampling point through the SDF model, calculates the target surface points based on the changes in the SDF value, and generates a shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image. The rendered ISAR image is compared with ISAR images from multiple views, and the SDF model is optimized in reverse using a loss function. The multi-resolution progressive optimization module is used to set multiple SDF model iterative optimization modules with different resolutions and iterative optimization parameters connected in series. The output of the previous SDF model iterative optimization module is processed and used as the input of the next SDF model iterative optimization module. The 3D reconstruction model output module is used to visualize the final SDF model output by the multi-resolution progressive optimization module and calculate the reconstruction accuracy as the 3D reconstruction result of the spatial target.
[0051] That is, the model initialization module is configured to execute step S1 to initialize the SDF model; The SDF model iterative optimization module is configured to execute steps S2, S3, and S4 to achieve SDF model iterative optimization. The multi-resolution progressive optimization module is connected to the model initialization module and is configured to execute step S5. It is responsible for configuring and updating the iterative optimization parameters during the reconstruction process and calling the SDF model iterative optimization module. The 3D reconstruction model output module is connected to the multi-resolution progressive optimization module and configured to execute step S6 to output the final reconstruction result and reconstruction accuracy.
[0052] The model initialization module specifically includes: (1a) Energy accumulation unit, used to calculate the energy accumulation value of each point uniformly sampled in a finite three-dimensional space from the input multi-view ISAR image; (1b) An adaptive threshold calculation unit, connected to the energy accumulation unit, is used to calculate a dynamic energy threshold based on the energy accumulation value of all current sampling points, instead of using a fixed threshold; (1c) Target point judgment unit, connected to energy accumulation unit and adaptive threshold calculation unit, used to determine whether the sampling point belongs to the target model based on the energy accumulation value and dynamic threshold; (1d) Point cloud conversion unit, connected to target point judgment unit, used to convert point clouds belonging to the target model into SDF models.
[0053] The SDF model iterative optimization module specifically includes: (2a) Shading image generation unit, which generates shading images based on the observation view and SDF model. It is a preprocessing unit before the ISAR image rendering unit, avoiding the observation view and SDF model being directly used as input to the ISAR image rendering unit. (2b) ISAR image rendering unit, connected to the Shading image generation unit, is used to obtain the rendered ISAR image based on the generated Shading image and the set neural network, so as to realize differentiable rendering; (2c) Loss function calculation unit, connected to the ISAR image rendering unit, is used to calculate the loss function value based on the rendered image, the input multi-view image, the SDF model, etc. (2d) SDF model optimization unit, connected to loss function calculation unit, used to optimize SDF model based on calculated loss function value.
[0054] The multi-resolution progressive optimization module specifically includes: (3a) Parameter setting unit, used to configure the SDF model resolution and the number and parameters of the SDF model iterative optimization modules; (3b) Upsampling unit, used to upsample the SDF model output by the SDF model iterative optimization module to a specified resolution; (3c) N SDF model iterative optimization modules, N 1.
[0055] The 3D reconstruction model output module specifically includes: (4a) SDF model visualization unit, used to convert the final SDF model from implicit representation to explicit representation, output including voxel model and mesh model; (4b) Reconstruction accuracy calculation unit, connected to SDF model visualization unit, is used to calculate the 3D IOU value between the output voxel model and the voxel model of the real target.
[0056] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images, characterized in that, include: Acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points; The projection position of the initial sampling point on the corresponding ISAR image is calculated based on the observation view of each ISAR image. The image pixel value of the projection position is used as the energy of the initial sampling point, and the SDF model is initialized according to the energy accumulation method. Three-dimensional spatial sampling points are generated based on the observation viewpoint and imaging parameters of each ISAR image. The SDF value of each three-dimensional spatial sampling point is calculated using the SDF model. The target surface points are calculated based on the changes in the SDF values to generate a Shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image; The rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is iteratively optimized in reverse using a loss function to obtain the final SDF model, which serves as the result of the three-dimensional reconstruction of the space target.
2. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The energy accumulation method includes: The energy of the initial sampling points on each ISAR image is accumulated, an adaptive threshold is calculated based on the maximum accumulated energy, and the initial sampling points with accumulated energy greater than the adaptive threshold are taken as target points, thereby constructing an initial point cloud; an initialized SDF model is constructed based on the initial point cloud.
3. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 2, characterized in that, The expression for the initial sampling point is: In the formula, It is the position index of a three-dimensional matrix. For resolution, It is the minimum boundary in three directions in three-dimensional space. For three-dimensional position The corresponding initial sampling point; The expression for the projection position of the initial sampling point on the corresponding ISAR image is: In the formula, Initial sampling point The row coordinates of the projection position on the m-th ISAR image. Initial sampling point The column coordinates of the projected position on the m-th ISAR image. and These are the height and width of the ISAR image, respectively. and These are the distance dimension resolution and the orientation dimension resolution, respectively. and These are the range dimension imaging axis and the azimuth dimension imaging axis, which are parameters belonging to the observation perspective; The formula for calculating the cumulative energy of the initial sampling points on each ISAR image is as follows: In the formula, Initial sampling point Cumulative energy on each ISAR image The total number of ISAR images. For ISAR images in Image pixel values at the location.
4. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The process of generating the three-dimensional spatial sampling points includes: The sampling start point and sampling direction are calculated based on the observation angle and imaging parameters. The corresponding calculation expressions are as follows: In the formula, The sampling start point, For the distance dimension imaging axis, As the azimuth imaging axis, The height of the imaging plane, The width of the imaging plane, For distance index, For direction index, For distance dimension resolution, For azimuth resolution, Sampling direction; From the starting point Begin, based on the given sampling range , ], along the sampling direction Sampling yields sampling points uniformly distributed in three-dimensional space. The corresponding calculation expression is: In the formula, Sampling direction The total number of samples.
5. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The calculation of target surface points based on changes in SDF values includes: Calculate the product of the SDF values of two adjacent 3D space sampling points. If the product is negative, select the point with the smaller SDF value from the two 3D space sampling points. Another point is The point is calculated using linear interpolation. and points New points between ; Calculate new points The SDF value is calculated and compared with a preset threshold. If it is less than the threshold, a new point is selected. As the target surface point; otherwise, the point ,point And new points Select two points with opposite signs and smaller SDF values as new points. and points Recalculate the new points until the new point The SDF value is less than the threshold, thus it is used as the target surface point.
6. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, In the process of generating a shading image from target surface points, the expression for calculating the pixel value of the shading image is as follows: In the formula, For the first shading image pixel value, For pixel index, For the first The number of target surface points corresponding to each pixel value For the first The corresponding pixel One target surface point, To obtain the maximum value, The direction of the incident electromagnetic wave, For calculations based on the unit normal, the corresponding expression is: In the formula, The SDF model matrix is... For normalization, It is a three-dimensional index of the SDF model matrix.
7. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The differentiable rendering network is a convolutional neural network; The loss function includes image rendering error loss, shape constraint loss, and regularization term.
8. The method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The SDF model is iteratively optimized using a multi-resolution progressive optimization strategy, which includes: Multiple SDF model iterative optimization modules with different resolutions and iterative optimization parameters are set up. The iterative optimization parameters include the iteration termination condition. The SDF model iterative optimization modules are connected in sequence. The final SDF model output by the previous SDF model iterative optimization module is upsampled or downsampled to a specified resolution and used as the input of the next SDF model iterative optimization module. The processing steps of each SDF model iterative optimization module include: Three-dimensional spatial sampling points are generated based on the observation viewpoint and imaging parameters of each ISAR image. The SDF value of each three-dimensional spatial sampling point is calculated using the SDF model. The target surface points are calculated based on the changes in the SDF values to generate a Shading image. The shading image is input into a trained differentiable rendering network to obtain a rendered ISAR image; The rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is back-optimized using a loss function to obtain the final SDF model.
9. A method for three-dimensional reconstruction of spatial targets based on multi-view ISAR images according to claim 1, characterized in that, The method further includes converting the final SDF model into a voxel model and a mesh model for visualization, and calculating the crossover ratio (CROR) between the converted voxel model and the voxel model of the actual target as an evaluation index of model reconstruction accuracy, so as to quantitatively evaluate the reconstruction accuracy and determine whether the preset reconstruction accuracy requirements are met.
10. A space target 3D reconstruction system that implements the space target 3D reconstruction method based on multi-view ISAR images as described in any one of claims 1-9, characterized in that, include: The model initialization module is used to acquire ISAR images from multiple perspectives, perform uniform sampling in a three-dimensional space with limited range, and determine the initial sampling points. The projection position of the initial sampling point on the corresponding ISAR image is calculated based on the observation view of each ISAR image. The image pixel value of the projection position is used as the energy of the initial sampling point, and the SDF model is initialized according to the energy accumulation method. The SDF model iterative optimization module is used to generate three-dimensional spatial sampling points based on the observation view and imaging parameters of each ISAR image, calculate the SDF value of each three-dimensional spatial sampling point through the SDF model, calculate the target surface points based on the changes in the SDF value, and generate a shading image. The shading image is input into the trained differentiable rendering network to obtain the rendered ISAR image; the rendered ISAR image is compared with the multi-view ISAR image, and the SDF model is back-optimized using the loss function; The multi-resolution progressive optimization module is used to set multiple SDF model iterative optimization modules with different resolutions and iterative optimization parameters connected in series. The output of the previous SDF model iterative optimization module is processed and used as the input of the next SDF model iterative optimization module. The 3D reconstruction model output module is used to visualize the final SDF model output by the multi-resolution progressive optimization module and calculate the reconstruction accuracy as the 3D reconstruction result of the spatial target.