Inverse synthetic aperture radar three-dimensional imaging method and system based on multilevel knowledge distillation

Abstract

The invention discloses an inverse synthetic aperture radar three-dimensional imaging method and system based on multilevel knowledge distillation. The method comprises the following steps: dividing an ISAR image and corresponding LOS data in orbit training data into a plurality of increment stages according to a time sequence; when the increment stage is a middle stage, obtaining a current ISAR image and current LOS data, and obtaining historical LOS data from a playback buffer area; through a multi-level knowledge distillation mechanism, a new model in the neural radiation field model is trained according to the current ISAR image, the current LOS data and the historical LOS data, and the multi-level knowledge distillation mechanism comprises pixel-level distillation, field-level distillation and feature-level distillation; and updating the playback buffer area through a data selection strategy to obtain a three-dimensional imaging model. The problem of disastrous forgetting in the continuous learning process is effectively relieved, the training efficiency and the imaging precision are improved, and the method can be widely applied to the technical field of radar imaging.

Description

Technical Field

[0001] The present application relates to the field of radar imaging technology, and in particular to a three-dimensional imaging method and system for inverse synthetic aperture radar based on multi-level knowledge distillation. Background Art

[0002] Inverse-Synthetic-Aperture-Radar (ISAR) uses the target's motion information relative to the radar to achieve all-weather, high-resolution target detection and recognition through multi-view imaging.

[0003] In recent years, three-dimensional imaging technology based on Neural Radiance Fields (NeRF) has developed rapidly. NeRF uses implicit neural networks to encode the three-dimensional information of the target, thereby achieving direct, high-quality three-dimensional reconstruction. Compared with traditional three-dimensional imaging methods (which usually require expensive hardware or rely on precise scattering point matching), NeRF technology can achieve geometrically consistent multi-view imaging using full-view training images while obtaining comprehensive view data. However, in practical applications, due to the limitations of single radar or simultaneous observations of multiple radars, ISAR images of the target are often acquired sequentially, making it difficult to achieve instant high-quality three-dimensional imaging.

[0004] To address the continuous learning challenges associated with incremental data acquisition, previous studies have attempted to incorporate continuous learning algorithms into NeRF imaging. For example, knowledge distillation techniques can be used to transfer information from the old model on previous data as a supervisory signal to the new model, mitigating the problem of catastrophic forgetting. However, existing continuous learning methods primarily rely on pixel-level knowledge distillation, which only replays two-dimensional image information. This single replay approach is insufficient to fully preserve the target's three-dimensional structure and can easily lead to catastrophic forgetting, causing the model to lose important previously learned information when new data is added.

[0005] In addition, when using experience replay technology, directly storing all historical data can easily lead to the accumulation of redundant line of sight (LOS) data, reduce training efficiency, and have a negative impact on imaging effects, thereby affecting the stability and overall performance of the model. Summary of the Invention

[0006] To solve the above technical problems, the purpose of the present invention is to provide an inverse synthetic aperture radar three-dimensional imaging method and system based on multi-level knowledge distillation, which can effectively deal with the problems caused by gradual data collection and information forgetting, and improve training efficiency.

[0007] To achieve the above objectives, an embodiment of the present application provides an inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation, comprising the following steps:

[0008] Acquire a neural radiation field model, wherein the neural radiation field model includes a trainable new model;

[0009] Acquire orbit training data, and divide the ISAR images and corresponding LOS data in the orbit training data into a plurality of incremental stages in chronological order;

[0010] When the incremental stage is an intermediate stage, obtaining a current ISAR image and current LOS data corresponding to the incremental stage, and obtaining historical LOS data from a preset playback buffer;

[0011] Training the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism;

[0012] The playback buffer is updated according to the current LOS data and the historical LOS data through a preset data selection strategy to obtain a three-dimensional imaging model;

[0013] Collecting three-dimensional coordinate points to be imaged, and obtaining a three-dimensional imaging result based on the three-dimensional coordinate points to be imaged using the three-dimensional imaging model;

[0014] Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation and feature-level distillation.

[0015] In some embodiments, the neural radiation field model further includes an old model in a frozen state, and the inverse synthetic aperture radar three-dimensional imaging method further includes:

[0016] When the incremental stage is the initial stage, obtaining the initial ISAR image and initial LOS data corresponding to the incremental stage;

[0017] The old model is trained according to the initial ISAR image and the initial LOS data.

[0018] In some embodiments, the training of the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism specifically includes:

[0019] Randomly sampling the historical LOS data to obtain historical playback data;

[0020] Randomly sampling the current LOS data to obtain current stage data;

[0021] Calculating pixel-level distillation loss, field-level distillation loss, and feature-level distillation loss based on the historical playback data and the current stage data;

[0022] The new model is trained according to the pixel-level distillation loss, the field-level distillation loss, and the feature-level distillation loss.

[0023] In some embodiments, the pixel-level distillation loss is calculated by the following formula:

[0024]

[0025] Among them, L pixel represents the pixel-level distillation loss, N L Represents the number of lines in the normal vector set L, l represents the normal vector of the imaging plane and l∈L, represents the complex predicted image of the l-th normal vector, s(l) represents the complex true value image of the l-th normal vector, ε represents the regularization constant, λ represents the weight coefficient, and sg(·) represents the stop gradient operation.

[0026] In some embodiments, the field-level distillation loss is calculated by the following formula:

[0027]

[0028] Among them, L field represents the field-level distillation loss, N represents the number of spatial points, i represents a spatial point and i∈N, Represents the backscattering coefficient of the i-th spatial point output by the new model, represents the backscattering coefficient of the i-th spatial point output by the old model, represents the particle density of the i-th spatial point output by the new model, represents the particle density of the i-th spatial point output by the old model, α and β represent hyperparameters, and MSE(·,·) represents the operation of calculating the mean square error.

[0029] In some embodiments, the characteristic distillation loss is calculated by the following formula:

[0030]

[0031] Among them, L feature represents the feature-level distillation loss, γ represents the hyperparameter, l represents the number of layers, CKA i represents the center kernel alignment between the features in the i-th layer, ||·|| F represents the Frobenius norm of the matrix, φ i (·) represents the new model θ t With the old model θ t-1 Features at layer i.

[0032] In some embodiments, updating the playback buffer according to the current LOS data and the historical LOS data using a preset data selection strategy to obtain a three-dimensional imaging model specifically includes:

[0033] Setting a minimum angle threshold between the current LOS data and the historical LOS data;

[0034] Calculating a minimum angle between the current LOS data and the historical LOS data, and determining whether the minimum angle is greater than a minimum angle threshold;

[0035] When the minimum angle is greater than the minimum angle threshold, storing the current LOS data in the playback buffer;

[0036] When the incremental stage is the termination stage, the training of the new model is stopped to obtain the three-dimensional imaging model.

[0037] To achieve the above objectives, another aspect of the present application provides an inverse synthetic aperture radar 3D imaging system based on multi-level knowledge distillation, comprising:

[0038] A first module is configured to obtain a neural radiation field model, wherein the neural radiation field model includes a trainable new model;

[0039] The second module is used to obtain orbit training data and divide the ISAR images and corresponding LOS data in the orbit training data into several incremental stages in chronological order;

[0040] A third module is configured to, when the incremental stage is an intermediate stage, obtain the current ISAR image and current LOS data corresponding to the incremental stage, and obtain historical LOS data from a preset playback buffer;

[0041] A fourth module is configured to train the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism;

[0042] A fifth module is configured to update the playback buffer according to the current LOS data and the historical LOS data using a preset data selection strategy to obtain a three-dimensional imaging model;

[0043] The sixth module is used to collect the three-dimensional coordinate points to be imaged, and obtain the three-dimensional imaging results according to the three-dimensional coordinate points to be imaged through the three-dimensional imaging model;

[0044] Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation and feature-level distillation.

[0045] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory, a processor, a program stored on the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation as described above is realized.

[0046] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation as described above.

[0047] The beneficial effects of the present invention are as follows: the inverse synthetic aperture radar three-dimensional imaging method and system based on multi-level knowledge distillation of the present invention, on the one hand, utilizes neural radiance field (NeRF) technology to realize three-dimensional reconstruction of space targets, and realizes comprehensive playback of pixel-level, field-level and feature-level information through a multi-level knowledge distillation mechanism, effectively alleviating the problem of catastrophic forgetting in the continuous learning process, so that the model can still maintain the complete reconstruction capability of the target three-dimensional structure when facing gradually collected data; on the other hand, an efficient memory data selection strategy is introduced, which can ensure the diversity of playback data on LOS and avoid redundant data interference, thereby improving training efficiency and imaging accuracy, and enhancing the stability and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0049] Figure 1 A flowchart of the steps of an inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation provided by one embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the process of step S102 provided in one embodiment of the present invention;

[0051] Figure 3 A schematic diagram of a process for training a three-dimensional imaging model according to an embodiment of the present invention;

[0052] Figure 4A core structure diagram of a CL-ISAR-NeRF system provided in one embodiment of the present invention;

[0053] Figure 5 A schematic diagram of a process for obtaining three-dimensional imaging results according to an embodiment of the present invention;

[0054] FIG6( a ) is a schematic diagram of a Tiangong-1 simulation data set provided by an embodiment of the present invention;

[0055] FIG6( b ) is a schematic diagram of an international space station simulation data set provided by an embodiment of the present invention;

[0056] Figure 7 A comparison chart of visualization results of new perspective synthesis and three-dimensional imaging using different methods provided in one embodiment of the present invention;

[0057] FIG8( a ) is a comparative diagram showing the evolution of new perspective synthesis performance during continuous learning using different methods provided by an embodiment of the present invention;

[0058] FIG8( b ) is a comparative diagram showing the evolution of new perspective synthesis performance during continuous learning using different methods provided in another embodiment of the present invention;

[0059] Figure 9 A schematic diagram of the structure of an inverse synthetic aperture radar 3D imaging system based on multi-level knowledge distillation provided by an embodiment of the present invention;

[0060] Figure 10 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0062] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0063] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0064] Before explaining the embodiments of the present application in detail, some of the nouns and terms involved in the embodiments of the present application are first explained. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.

[0065] Neural Radiance Fields (NeRF): A deep learning model for 3D scene reconstruction. It is trained to learn how light in a scene interacts with the surface of an object, thereby generating high-quality 3D scene images.

[0066] Inverse Synthetic Aperture Radar (ISAR) is a radar technology that generates high-resolution images of a target by analyzing the reflections of radar waves from the target. ISAR utilizes the relative motion of the radar platform during flight or movement to simulate a larger antenna aperture, thereby improving imaging resolution.

[0067] Line of Sight (LOS): In radar field, it refers to the direction of the direct line between the radar and the target.

[0068] Peak Signal to Noise Ratio (PSNR): An evaluation indicator used to assess the difference between two images.

[0069] Structural Similarity Index Measure (SSIM): is a metric used to quantify the structural similarity between two images. It implements structural similarity calculation based on a model of the human visual system and is very sensitive to the perception of local structural changes in images.

[0070] Simplified General Perturbations-4 (SGP4): A model used to predict the position and velocity of Earth-orbiting satellites. It is based on two-row element data of the satellite and takes into account factors such as atmospheric drag and the non-spherical shape of the Earth to perform relatively accurate calculations of satellite orbits.

[0071] Catastrophic Forgetting (CF): refers to the phenomenon that when an artificial intelligence system (such as a deep learning model) learns a new task or adapts to a new environment, the previously learned knowledge is overwritten or forgotten due to the parameter update mechanism.

[0072] Continual Learning (CL) is a machine learning paradigm that aims to enable models to achieve long-term adaptation and performance improvement by dynamically introducing new data or tasks without forgetting old knowledge.

[0073] Inverse Synthetic Aperture Radar (ISAR) utilizes information about the target's motion relative to the radar to achieve all-weather, high-resolution target detection and recognition through multi-view imaging. The core advantage of traditional ISAR imaging lies in its ability to capture high-quality two-dimensional images, unaffected by lighting and weather conditions. While two-dimensional ISAR images are somewhat effective in target recognition, these two-dimensional images often fail to fully reflect the target's spatial structure and geometry. Three-dimensional ISAR imaging can extract high-dimensional information about the target, such as its spatial geometry, size, and other key features. This is crucial for space target monitoring, satellite health assessment, and prediction of future target motion.

[0074] In recent years, three-dimensional imaging technology based on Neural Radiance Fields (NeRF) has developed rapidly. NeRF uses implicit neural networks to encode the three-dimensional information of the target, thereby achieving direct, high-quality three-dimensional reconstruction. Compared with traditional three-dimensional imaging methods (which usually require expensive hardware or rely on precise scattering point matching), NeRF technology can achieve geometrically consistent multi-view imaging using full-view training images while obtaining comprehensive view data. However, in practical applications, due to the limitations of single radar or simultaneous observations of multiple radars, ISAR images of the target are often acquired sequentially, making it difficult to achieve instant high-quality three-dimensional imaging.

[0075] To address the continuous learning challenges associated with incremental data acquisition, previous studies have attempted to incorporate continuous learning algorithms into NeRF imaging. For example, knowledge distillation techniques can be used to transfer information from the old model on previous data as a supervisory signal to the new model, mitigating the problem of catastrophic forgetting. However, existing continuous learning methods primarily rely on pixel-level knowledge distillation, which only replays two-dimensional image information. This single replay approach is insufficient to fully preserve the target's three-dimensional structure and can easily lead to catastrophic forgetting, causing the model to lose important previously learned information when new data is added.

[0076] In addition, when using experience replay technology, directly storing all historical data can easily lead to the accumulation of redundant line of sight (LOS) data, reduce training efficiency, and have a negative impact on imaging effects, thereby affecting the stability and overall performance of the model.

[0077] To this end, an embodiment of the present invention proposes an inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation. On the one hand, neural radiance field (NeRF) technology is used to realize 3D reconstruction of spatial targets, and a multi-level knowledge distillation mechanism is used to achieve comprehensive playback of pixel-level, field-level, and feature-level information, effectively alleviating the problem of catastrophic forgetting in the continuous learning process, so that the model can still maintain the complete reconstruction capability of the target 3D structure when facing gradually collected data; on the other hand, an efficient memory data selection strategy is introduced to ensure the diversity of the playback data on the LOS and avoid redundant data interference, thereby improving training efficiency and imaging accuracy, and enhancing the stability and adaptability of the system.

[0078] Reference Figure 1 , Figure 1 A flowchart of the steps of an inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation provided by an embodiment of the present invention. The embodiment of the present invention provides an inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation, which includes steps S101 to S106:

[0079] S101. Obtain a neural radiation field model, where the neural radiation field model includes a trainable new model;

[0080] Specifically, the neural radiation field model is divided into two parts, one is the old model θ in the frozen state t-1 , used to generate playback information of historical knowledge, and the other part is a new trainable model θ t .

[0081] S102, obtaining orbit training data, and dividing the ISAR images and corresponding LOS data in the orbit training data into several incremental stages in chronological order;

[0082] Specifically, if Figure 2The flowchart of step S102 is shown as follows. First, the real track training data is obtained. In the track training data, the three-dimensional ECEF coordinates of the radar and the three-dimensional ECEF coordinates of the target are calculated. Then, based on the three-dimensional ECEF coordinates of the radar and the target, the imaging trajectory is obtained, and the LOS data (i.e., radar line of sight data) is calculated. Then, the radar echo data is received, and the radar echo data is Doppler processed to generate an ISAR image (i.e., an inverse synthetic aperture radar image). Finally, the ISAR image and the corresponding LOS data are divided into several incremental stages 0-t in chronological order.

[0083] As an optional implementation, the neural radiation field model further includes an old model in a frozen state, and the inverse synthetic aperture radar three-dimensional imaging method further includes the following steps S1021 and S1022:

[0084] S1021. When the incremental stage is the initial stage, obtain the initial ISAR image and initial LOS data corresponding to the incremental stage;

[0085] S1022: Train the old model based on the initial ISAR image and the initial LOS data.

[0086] Specifically, if Figure 3 The figure shows the process flow of 3D imaging model training. In the initial stage 0, the initial ISAR image P0 and initial LOS data R0 corresponding to the initial stage 0 are collected, and the difference L between the real ISAR image and the NeRF rendered image is minimized. ISAR-NeRF Train the old model θ t-1 .

[0087] S103, when the incremental stage is an intermediate stage, obtaining the current ISAR image and current LOS data corresponding to the incremental stage, and obtaining historical LOS data from a preset playback buffer;

[0088] Specifically, if Figure 3 As shown, at the intermediate stage t, the current ISAR image P corresponding to the intermediate stage t is collected t and current LOS data R t , playback buffer M t-1 Stores the diverse LOS data {R'0, R'1, ... R' t-1}, get the historical LOS data {R'0, R'1, ... R' in the playback buffer t-1}.

[0089] S104, training the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism;

[0090] Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation, and feature-level distillation.

[0091] It should be noted that the embodiment of the present invention simultaneously replays information at the pixel level, field level (backscattering coefficient and particle density) and feature level (deep features extracted using central kernel alignment technology), which can make full use of the multi-level information obtained in the previous task. Through this multi-level knowledge distillation mechanism, it can effectively alleviate the catastrophic forgetting problem in the continuous learning process, ensuring that the new model can completely retain the previously learned three-dimensional structure information when introducing new data, avoiding the model from forgetting due to using only current data.

[0092] As an optional implementation, step S104 may be specifically divided into the following steps S1041 to S1044:

[0093] S1041. Randomly sample historical LOS data to obtain historical playback data;

[0094] S1042. Randomly sample the current LOS data to obtain current stage data;

[0095] S1043. Calculate pixel-level distillation loss, field-level distillation loss, and feature-level distillation loss based on historical playback data and current stage data.

[0096] S1044. Train the new model based on pixel-level distillation loss, field-level distillation loss, and feature-level distillation loss.

[0097] In some optional embodiments, such as Figure 4 The figure shows the core structure of the CL-ISAR-NeRF system. The system achieves continuous incremental 3D imaging by simultaneously considering pixel-level, field-level, and feature-level information, overcoming the catastrophic forgetting problem in continuous learning scenarios. Figure 3 and Figure 4 As shown, in each training iteration, the playback buffer M t-1 and current LOS data R t Uniformly sample LOS data χ = χ GR ∪χ t , where χ GR Represents historical playback data, χ t Represents the current stage data, and the sampled data is used to update the model θ t . Then calculate the pixel-level distillation loss L pixel , field-level distillation loss L field and characteristic distillation loss L feature , according to the pixel-level distillation loss L pixel , field-level distillation loss L field and characteristic distillation loss Lfeature The weighted sum of the new model θ t Conduct training.

[0098] Specifically, pixel-level distillation supervises the new model using the original image; field-level distillation uses the mean square error (MSE) constraint between the backscattering coefficient ρ and the particle density σ; and feature-level distillation uses the Central Kernel Alignment (CKA) metric to measure the similarity of intermediate-level features. By comprehensively replaying information at different levels, the system can fully supplement previously learned information with a diverse selection of historical LOS data, thereby improving the new model's ability to retain existing knowledge and adapt to new information.

[0099] As an optional implementation, the pixel-level distillation loss is calculated as follows:

[0100]

[0101] Among them, L pixel represents the pixel-level distillation loss, N L Represents the number of lines in the normal vector set L, l represents the normal vector of the imaging plane and l∈L, represents the complex predicted image of the l-th normal vector, s(l) represents the complex true value image of the l-th normal vector, ε represents the regularization constant, λ represents the weight coefficient, and sg(·) represents the stop gradient operation.

[0102] Specifically, we use pixel-level distillation loss L pixel Playback pixel-level information, for the current stage data χ t : Directly use the real ISAR image P t As a supervisory signal; for historical playback data χ GR :Use the old model θ t-1 The rendered image serves as a supervisory signal, constraining the new model output from a pixel-wise perspective. This strategy stabilizes the model's image rendering performance under previous views and serves as a basic playback method.

[0103] As an optional embodiment, the field-level distillation loss is calculated by the following formula:

[0104]

[0105] Among them, L field represents the field-level distillation loss, N represents the number of spatial points, i represents a spatial point and i∈N, Represents the backscattering coefficient of the i-th spatial point output by the new model, represents the backscattering coefficient of the i-th spatial point output by the old model, represents the particle density of the i-th spatial point output by the new model, represents the particle density of the i-th spatial point output by the old model, α and β represent hyperparameters, and MSE(·,·) represents the operation of calculating the mean square error.

[0106] Specifically, in order to fully maintain the radiation field information of the model, for the historical playback data χ GR , loss L through field-level distillation field The outputs of the new and old models on the backscattering coefficient ρ and particle density σ are calculated respectively. The hyperparameters α and β are used to control the weights of different information. The mean square error (MSE) metric is used to ensure that the new model can fully inherit the radiation field information, thereby maintaining the three-dimensional imaging accuracy.

[0107] As an optional embodiment, the characteristic distillation loss is calculated by the following formula:

[0108]

[0109] Among them, L feature represents the feature-level distillation loss, γ represents the hyperparameter, l represents the number of layers, and CKA i represents the center kernel alignment between the features in the i-th layer, ||·|| F represents the Frobenius norm of the matrix, φ i (·) represents the new model θ t With the old model θ t-1 Features at layer i.

[0110] Specifically, considering that the intermediate layer features contain a large amount of discriminative information, the embodiment of the present invention introduces a feature-level distillation loss L based on center kernel alignment (CKA) feature The CKA value between features at each layer is defined by the following formula:

[0111]

[0112] Among them CKA i The larger the value of ∈[0,1], the higher the similarity between the two features. The above feature-level distillation loss L is constructed after averaging each layer. feature .

[0113] S105, updating the playback buffer according to the current LOS data and the historical LOS data through a preset data selection strategy to obtain a three-dimensional imaging model;

[0114] It should be noted that ISAR-NeRF requires a long period of data accumulation to achieve high performance, which is not realistic in practical applications. Therefore, the goal is to achieve incremental 3D imaging during the data acquisition process, that is, continuous learning setting. In this scenario, data {D0, D1, ..., D t ,...}. When new images appear, the available 3D imaging results need to be provided immediately and continuously improved in subsequent data. Consistent with existing work, at stage t, it is assumed that the currently available ISAR images are limited to the current stage data, while the LOS data are not restricted, that is, the model θ t Accessible P t ∪{R0,R1,...,R t The storage cost of image data is much higher than that of LOS (a three-dimensional vector), so the embodiment of the present invention uses an efficient memory data selection strategy to restrict image data to prevent redundancy of LOS data stored in the playback buffer and improve its diversity.

[0115] As an optional implementation, step S105 may be specifically divided into the following steps S1051 to S1054:

[0116] S1051. Set a minimum angle threshold between the current LOS data and the historical LOS data;

[0117] S1052: Calculate the minimum angle between the current LOS data and the historical LOS data, and determine whether the minimum angle is greater than a minimum angle threshold;

[0118] S1053: When the minimum angle is greater than the minimum angle threshold, the current LOS data is stored in the playback buffer;

[0119] S1054: When the incremental stage is the termination stage, the training of the new model is stopped to obtain a three-dimensional imaging model.

[0120] Specifically, the strategy is based on calculating the minimum angle between the selected LOS data (i.e., the current LOS data) and the existing LOS data in the playback buffer (i.e., the historical LOS data), using the following judgment conditions:

[0121]

[0122] Among them, LOS j is the selected data in the playback buffer (i.e., historical LOS data), θ th is the set minimum angle threshold. When the minimum angle between the selected LOS data and the existing LOS data is greater than θ thThis strategy ensures that the system replays historical information from diverse LOS, effectively improving model rendering and 3D reconstruction accuracy while avoiding the reduction of training efficiency due to the accumulation of similar data.

[0123] When all the data are trained, a three-dimensional imaging model is obtained that can realize real-time incremental three-dimensional imaging.

[0124] S106 , collecting the three-dimensional coordinate points to be imaged, and obtaining a three-dimensional imaging result based on the three-dimensional coordinate points to be imaged using a three-dimensional imaging model.

[0125] Specifically, if Figure 5 Figure 2 shows a schematic diagram of the process for acquiring 3D imaging results. First, a set of discrete 3D coordinate points are collected in the space to be imaged. Then, based on the NeRF grid parameters, the voxel density value of each sampling point is predicted to reflect the geometric structure. The density of all sampling points is aggregated to form a "voxel density" field. Then, the Marching Cubes algorithm is used to extract isosurfaces from the "voxel density" field to generate the geometric outline of the target. Finally, the grid and view angle (LOS) are input into the 3D imaging model for rendering to obtain a visual 3D imaging result.

[0126] The above describes the inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to an embodiment of the present invention. It can be seen that compared with the prior art, the present invention has the following advantages:

[0127] First, a continuous learning architecture was constructed for sequentially arriving ISAR image data in practical applications. This architecture can generate high-quality 3D imaging results in a timely manner at each data stage, while continuously optimizing and improving the imaging results with subsequent data, achieving real-time incremental 3D imaging.

[0128] Second, a multi-level knowledge distillation mechanism enables comprehensive replay of pixel-level, field-level, and feature-level information, effectively alleviating the problem of catastrophic forgetting during continuous learning. This allows the model to maintain the ability to fully reconstruct the target's 3D structure even when faced with incrementally acquired data. Furthermore, this technology can rapidly generate high-quality imaging results in real-time incremental 3D imaging scenarios, meeting the needs of actual monitoring and dynamic updates.

[0129] Third, we introduced an efficient memory data selection strategy. By controlling the minimum angular difference between lines of sight (LOS) in the replayed data, we effectively reduced the interference of redundant data, improving training efficiency and imaging accuracy. This approach, within a continuous learning framework, balances model stability and scalability, significantly surpassing existing technologies in terms of knowledge retention and real-time data processing.

[0130] To further verify the accuracy of the embodiment of the present invention, the system model performance is verified in combination with simulation experiments to further illustrate the effects of the embodiment of the present invention.

[0131] The experiment was conducted on the Tiangong-1 and International Space Station datasets. The construction of the Tiangong-1 simulation dataset is shown in Figure 6(a), and the construction of the International Space Station simulation dataset is shown in Figure 6(b). In the five incremental stages, five different locations were randomly selected to deploy radars, and their corresponding Earth-centered Earth-fixed coordinates (x n ,y n ,z n ). The Earth-centered Earth-fixed coordinates of the satellite (x s ,y s ,z s ) is obtained through the Simplified General Perturbations-4 (SGP4) model, and the trajectory of the imaging plane in each stage is calculated. Each stage contains 62 training images and 62 test images.

[0132] Figures 6(a) and 6(b) show schematic diagrams of a serialized simulation dataset designed based on the SGP4 model. Rectangles of different colors in the figure represent imaging planes generated by different radars at different locations. Each imaging plane corresponds to a radar observation position, and the target spatial position is precisely calculated during the simulation. The dataset exhibits a multi-stage, continuous incremental nature, with each stage containing a certain number of training and test ISAR images and corresponding LOS data. This architecture ensures that the model can fully simulate actual continuous learning scenarios in a simulation environment, verifying the model's performance improvements in 3D imaging and novel perspective synthesis tasks under different radar perspectives.

[0133] Table 1 below shows the quantitative results of the final model on new perspective synthesis after five incremental stages, and Table 2 below shows the quantitative results of the final model on three-dimensional imaging tasks after five incremental stages. It can be seen from Tables 1 and 2 below that the embodiments of the present invention outperform all baseline methods on both datasets. Among them, the elastic weight fixing method performs poorly in continuous learning tasks due to the constraints on the model's expressiveness; the experience replay method is limited in performance due to the limited replay buffer capacity; although the generative replay method improves stability through pixel-level distillation, it fails to fully utilize existing information and data selection, and its performance is limited. The embodiment of the present invention better balances the plasticity and stability of the model by simultaneously performing distillation at the pixel, field, and feature levels, combined with efficient memory data selection.

[0134] Table 1

[0135]

[0136] Table 2

[0137]

[0138]

[0139] like Figure 7 The comparison of visualization results of different methods in new perspective synthesis and three-dimensional imaging is shown. Figure 7 It can be seen that the embodiment of the present invention has a high consistency with the true value in maintaining the strong scattering area, while ensuring the integrity and accuracy of three-dimensional imaging; while some comparison methods have obvious deviations in image rendering and three-dimensional imaging.

[0140] Figures 8(a) and 8(b) illustrate the evolution of novel perspective synthesis performance across different methods during continuous learning. The models were evaluated using test data from stage 0 to stage t. As shown in Figures 8(a) and 8(b), the embodiments of the present invention outperform all baseline methods at each incremental stage. The evolution of novel perspective synthesis performance demonstrates the superiority of the proposed methods in maintaining high-quality image generation.

[0141] Reference Figure 9 The embodiment of the present invention further provides an inverse synthetic aperture radar three-dimensional imaging system based on multi-level knowledge distillation, comprising:

[0142] The first module is used to obtain a neural radiation field model, which includes a trainable new model;

[0143] The second module is used to obtain orbit training data and divide the ISAR images and corresponding LOS data in the orbit training data into several incremental stages in chronological order;

[0144] The third module is used for obtaining the current ISAR image and current LOS data corresponding to the incremental stage when the incremental stage is an intermediate stage, and obtaining historical LOS data from a preset playback buffer;

[0145] The fourth module is used to train the new model based on the current ISAR image, current LOS data, and historical LOS data through a multi-level knowledge distillation mechanism;

[0146] The fifth module is used to update the playback buffer according to the current LOS data and the historical LOS data through a preset data selection strategy to obtain a three-dimensional imaging model;

[0147] The sixth module is used to collect the three-dimensional coordinate points to be imaged, and obtain the three-dimensional imaging results according to the three-dimensional coordinate points to be imaged through the three-dimensional imaging model;

[0148] Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation, and feature-level distillation.

[0149] The contents of the above-mentioned embodiment of the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation are all applicable to the embodiment of the present inverse synthetic aperture radar three-dimensional imaging system based on multi-level knowledge distillation. The functions specifically implemented by the present embodiment of the inverse synthetic aperture radar three-dimensional imaging system based on multi-level knowledge distillation are the same as those of the above-mentioned embodiment of the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation, and the beneficial effects achieved are also the same as the beneficial effects achieved by the above-mentioned embodiment of the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation.

[0150] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communicating between the processor and the memory. When the program is executed by the processor, the aforementioned inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0151] like Figure 10 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 10 , an embodiment of the present invention provides an electronic device, including:

[0152] The processor 1001 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0153] Memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 1002 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in memory 1002 and is called by processor 1001 to execute the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation according to the embodiment of the present invention.

[0154] Input / output interface 1003, used to implement information input and output;

[0155] Communication interface 1004, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0156] Bus 1005 , which transmits information between various components of the device (e.g., processor 1001 , memory 1002 , input / output interface 1003 , and communication interface 1004 );

[0157] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0158] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation.

[0159] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0160] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0161] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0162] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0163] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0164] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0165] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0166] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0167] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0168] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0169] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for inverse synthetic aperture radar 3D imaging based on multi-level knowledge distillation, characterized in that: The following steps are involved: Acquire a neural radiation field model, wherein the neural radiation field model includes a trainable new model; Acquire orbit training data, and divide the ISAR images and corresponding LOS data in the orbit training data into a plurality of incremental stages in chronological order; When the incremental stage is an intermediate stage, obtaining a current ISAR image and current LOS data corresponding to the incremental stage, and obtaining historical LOS data from a preset playback buffer; Training the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism; The playback buffer is updated according to the current LOS data and the historical LOS data through a preset data selection strategy to obtain a three-dimensional imaging model; Collecting three-dimensional coordinate points to be imaged, and obtaining a three-dimensional imaging result based on the three-dimensional coordinate points to be imaged using the three-dimensional imaging model; Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation and feature-level distillation.

2. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to claim 1, characterized in that: The neural radiation field model also includes an old model in a frozen state, and the inverse synthetic aperture radar three-dimensional imaging method further includes: When the incremental stage is the initial stage, obtaining the initial ISAR image and initial LOS data corresponding to the incremental stage; The old model is trained according to the initial ISAR image and the initial LOS data.

3. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to claim 1, characterized in that: The training of the new model according to the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism specifically includes: Randomly sampling the historical LOS data to obtain historical playback data; Randomly sampling the current LOS data to obtain current stage data; Calculating pixel-level distillation loss, field-level distillation loss, and feature-level distillation loss based on the historical playback data and the current stage data; The new model is trained according to the pixel-level distillation loss, the field-level distillation loss, and the feature-level distillation loss.

4. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to claim 3, characterized in that: The pixel-level distillation loss is calculated as follows: Among them, L pixel represents the pixel-level distillation loss, N L Represents the number of lines in the normal vector set L, l represents the normal vector of the imaging plane and l∈L, represents the complex predicted image of the l-th normal vector, s(l) represents the complex true value image of the l-th normal vector, ε represents the regularization constant, λ represents the weight coefficient, and sg(·) represents the stop gradient operation.

5. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to claim 3, characterized in that: The field-level distillation loss is calculated by the following formula: Among them, L field represents the field-level distillation loss, N represents the number of spatial points, i represents a spatial point and i∈N, Represents the backscattering coefficient of the i-th spatial point output by the new model, represents the backscattering coefficient of the i-th spatial point output by the old model, represents the particle density of the i-th spatial point output by the new model, represents the particle density of the i-th spatial point output by the old model, α and β represent hyperparameters, and MSE(·,·) represents the operation of calculating the mean square error.

6. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to claim 3, characterized in that: The characteristic distillation loss is calculated by the following formula: Among them, L feature represents the feature-level distillation loss, γ represents the hyperparameter, l represents the number of layers, CKA i represents the center kernel alignment between the features in the i-th layer, ||·|| F represents the Frobenius norm of the matrix, φ i (·) represents the new model θ t With the old model θ t-1 Features at layer i.

7. The inverse synthetic aperture radar 3D imaging method based on multi-level knowledge distillation according to any one of claims 1 to 6, characterized in that: The updating of the playback buffer according to the current LOS data and the historical LOS data by a preset data selection strategy to obtain a three-dimensional imaging model specifically includes: Setting a minimum angle threshold between the current LOS data and the historical LOS data; Calculating a minimum angle between the current LOS data and the historical LOS data, and determining whether the minimum angle is greater than a minimum angle threshold; When the minimum angle is greater than the minimum angle threshold, storing the current LOS data in the playback buffer; When the incremental stage is the termination stage, the training of the new model is stopped to obtain the three-dimensional imaging model.

8. An inverse synthetic aperture radar three-dimensional imaging system based on multi-level knowledge distillation, characterized in that: include: A first module is configured to obtain a neural radiation field model, wherein the neural radiation field model includes a trainable new model; The second module is used to obtain orbit training data and divide the ISAR images and corresponding LOS data in the orbit training data into several incremental stages in chronological order; A third module is configured to, when the incremental stage is an intermediate stage, obtain the current ISAR image and current LOS data corresponding to the incremental stage, and obtain historical LOS data from a preset playback buffer; A fourth module is configured to train the new model based on the current ISAR image, the current LOS data, and the historical LOS data through a multi-level knowledge distillation mechanism; A fifth module is configured to update the playback buffer according to the current LOS data and the historical LOS data using a preset data selection strategy to obtain a three-dimensional imaging model; The sixth module is used to collect the three-dimensional coordinate points to be imaged, and obtain the three-dimensional imaging results according to the three-dimensional coordinate points to be imaged through the three-dimensional imaging model; Among them, the multi-level knowledge distillation mechanism includes pixel-level distillation, field-level distillation and feature-level distillation.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation as described in any one of claims 1 to 7 are implemented.

10. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the inverse synthetic aperture radar three-dimensional imaging method based on multi-level knowledge distillation as described in any one of claims 1 to 7.

Images