Low-forgetting through-the-wall radar high-resolution imaging method

Abstract

The invention discloses a low-forgetting through-the-wall radar high-resolution imaging method. The method comprises the following specific steps: training an optical image-assisted high-resolution imaging network of a source domain so that the optical image-assisted high-resolution imaging network can recover the shape of a source domain target and takes the optical image-assisted high-resolution imaging network as a pre-training network; a pre-training model is loaded to initialize corresponding part parameters of a target domain network LFHRNet with a multi-branch structure, then training is performed on the target domain network LFHRNet on a target domain data set, one branch learns new knowledge of a target domain during training, the other branch is frozen to reserve source domain knowledge, and a routing network is introduced to perform accurate expert allocation. According to the method, forgetting of source domain knowledge is effectively reduced, and the network has the capability of performing high-resolution imaging on static targets in the source domain and the target domain at the same time.

Description

Technical Field

[0001] The present invention belongs to the technical field of radar signal processing, and particularly relates to a high-resolution imaging method for through-wall radar with less forgetting. Background Art

[0002] With the continuous advancement of urbanization, the demand for detecting shielded spaces has increased significantly, playing an important role in the fields of security, rescue, medical treatment, etc.

[0003] Through-wall radar (TWR) uses low-frequency electromagnetic waves to penetrate buildings and achieve imaging display of shielded space targets, which is the main technical means for detecting shielded spaces.

[0004] Currently, for traditional through-wall radar imaging algorithms, due to the limitations of antenna aperture and array spacing, regardless of the shape of the target to be imaged, the imaging results are all in the form of blurred light spots, unable to directly reflect the target shape information and difficult to directly identify and use. Deep learning models cannot be generalized to new targets with large morphological differences, and it is difficult to collect enough labeled data in reality for their training. Using transfer learning can improve the imaging effect of the few-shot target domain, but the network forgets the source domain knowledge. As shown in Figure 1 the network no longer has the ability to correctly restore the target shape in the source domain. Therefore, reducing the network's forgetting of source domain knowledge and enabling the network to have the ability to perform high-resolution imaging of targets in both the source domain and the target domain is an urgent problem to be solved in this field. Summary of the Invention

[0005] The present invention proposes a high-resolution imaging method for through-wall radar with less forgetting. Aiming at the problems of insufficient imaging resolution and the network forgetting source domain knowledge after fine-tuning in the target domain, the present invention designs a less-forgetting high-resolution imaging network (LFHRNet) for the target domain, which is specifically a network with a multi-branch structure. The present invention first trains an optically assisted high-resolution imaging network in the source domain to enable it to restore the shape of the source domain target and use it as a pre-trained network. For the training of the target domain network LFHRNet, one branch learns new knowledge in the target domain, one branch remains unchanged to retain source domain knowledge, and a routing network is introduced for expert allocation. Finally, the forgetting of source domain knowledge is reduced, enabling the network to restore the shape of the target domain target while also being able to restore the shape of the source domain target.

[0006] The technical solution of the present invention is as follows:

[0007] A high-resolution imaging method for through-wall radar with less forgetting, comprising the following steps:

[0008] Step 1: Obtain the BP radar imaging results and the corresponding optical images, and construct a dataset:

[0009] Consider, for example, Figure 3 an imaging scenario of size N x ×N y containing a single reflection. The radar transmitted signal s(t) is a stepped-frequency signal with a starting frequency of f0, a stepped-frequency interval of Δf, and containing K frequency points. Then the frequencies within the bandwidth are:

[0010] f k = f0 + kΔf, k = 0,..., K - 1.

[0011] Assume the wall thickness is d wall , the dielectric constant is ε r , and the antenna is moved at an interval of d1. Then the m-th antenna position is (x m , -d1). Considering the direct reflection and single-reflection signals of the target, the wall direct-reflection signal, and the reflection signal between targets, the received echo signal at the m-th antenna position and the k-th frequency can be regarded as the sum of the target reflection signal the wall direct-reflection signal the reflection signal between targets and noise v m,k , and can be expressed as:

[0012]

[0013] where P is the number of targets, W is the number of wall direct-reflection paths, R is the number of multiple single-reflection paths on the target, σ p and σ pq are the reflection coefficients of the w-th wall direct-reflection path, the p-th target, and the path between the p-th and q-th targets, respectively. Taking Path-A shown in Figure 3 as an example, τ pm is the round-trip delay from the m-th antenna to the p-th target, is the round-trip delay of the w-th wall direct-reflection path, and can be expressed as:

[0014]

[0015] where l1 is the propagation distance of the electromagnetic wave in the wall, l2 and l3 are the propagation distances of the electromagnetic wave in the air, is the length of the w-th wall direct-reflection path, and v is the propagation speed of the electromagnetic wave in the wall. According to Snell's law, v can be expressed as:

[0016]

[0017] where c is the speed of light.

[0018] For three-dimensional BP imaging, the imaging region is divided into grids along the x, y, and z axes, and the echo data of M groups of antennas at different positions are collected. The echo data is subjected to IFFT transformation to obtain s(m,k), and the two-way time delay τ between the m-th antenna position and the h-th pixel in the imaging region is calculated. mh , then the imaging result of the h-th pixel is:

[0019]

[0020] Calculate the imaging of all pixels for the entire region.

[0021] Repeat the above steps, collect the echo data of different targets at different rotation angles and perform imaging to construct a complete simulation data set. And conduct actual measurement experiments to construct a complete actual measurement data set.

[0022] Step 2: Build and train a high-resolution imaging network assisted by source domain optical images:

[0023] The present invention uses conditional generative adversarial networks (cGANs) for high-resolution imaging. cGANs consist of a generator and a discriminator, as Figure 4 shown. The input to the network is a 3D radar image of size 32×32×32 and a 2D label of size 128×128.

[0024] The generator contains an encoder and a decoder. The encoder-decoder structure can extract abstract features from the input 3D radar image and use the decoder to reconstruct a high-quality 2D image from this compressed representation. The encoder consists of 5 sequentially connected 3D convolutional layers, which can map the 3D radar image into a vector of 2048×1. After each 3D convolutional layer, there are a BatchNorm layer and a LeakyReLU layer. The BatchNorm layer is for stabilizing training and improving the convergence of the network, and the LeakyReLU layer introduces non-linearity to prevent gradient vanishing or descent. The decoder consists of 5 sequentially connected 2D transposed convolutional layers, BatchNorm layers, and ReLU layers, which upsample the vector to reconstruct an image of 128×128. At the same time, in order to retain high-frequency information, the network uses skip connections, splices the generated image output by the decoder with the high-frequency channels of the radar image, and then passes through 2 transposed convolutional layers to obtain the final output.

[0025] The discriminator takes the 3D radar image and the 2D label or the generated image as inputs simultaneously, and uses two different networks to transform both into vectors. The structure of the 3D input end is the same as that of the encoder of the generator, and the 2D input end consists of 8 sequentially connected 2D convolutional layers, BatchNorm layers, and LeakyReLU layers. The outputs of both are concatenated into a vector and classified through a Linear layer. The classification result D(x,y) or D(x,G(x)) represents the probability that the 2D input is a real label. During the training phase, the discriminator is used to distinguish the real label y and the generated image G(x), and the generator is used to generate an image that meets the conditional constraints, making it as indistinguishable as possible to the discriminator. This is a game process, and the loss function of this process can be expressed as:

[0026]

[0027] To make the generated image have a better effect, the present invention introduces the root mean square error (MSE) and perceptual loss when training the cGAN network. Then, the loss function of the generator during the training process is:

[0028] L(G) = L GAN (G) + λ MSE L MSE + λ p L p

[0029] Among them, L GAN (G) is the adversarial loss of the generative adversarial network itself, L MSE is the MSE loss between the real label and the generated image to ensure the accuracy and consistency of the generated image and the label at the pixel level. L p is the perceptual loss calculated through the pre-trained VGG network to make the generated image more in line with human perception. λ MSE and λ p are constants, making the generated image correct in details and directly recognizable by the human eye at the same time. During training, the generator is optimized to maximize the probability that the generated image is classified as a real label and minimize L MSE and L p , and the discriminator is optimized to minimize the classification error probability between the generated image and the real label.

[0030] Step 3: Build and train the target domain network LFHRNet:

[0031] The structure of the target domain network LFHRNet designed by the present invention is as Figure 5As shown, the target domain network has the same encoder structure as the source domain network and integrates two parallel branches as experts for handling different tasks. The tasks in the source domain and the target domain respectively refer to high-resolution imaging of the source domain input and high-resolution imaging of the target domain input. When training the target domain network on target domain data, one branch is initialized with the source domain network and frozen during training to retain source domain knowledge, while the other expert's parameters are updated to learn new knowledge in the target domain. The training objective can be expressed as:

[0032]

[0033] where D t is the target domain dataset, containing 3D radar images x and 2D labels y. Θ encoder are the parameters pre-trained in the source domain, containing the knowledge learned in the source domain. The encoder of the target domain network is initialized with Θ encoder and frozen during training on D t to serve as an off-the-shelf feature extractor. ΔW are the parameters learned on the target domain data, and the optimal ΔW is obtained by minimizing the loss L t .

[0034] To integrate the two experts into one network, the present invention introduces a routing network to achieve the division of labor and precise allocation of the experts. When a 3D radar image is input into the target domain network, the 3D radar image may come from the source domain or the target domain. Due to the different data characteristics and distributions of the source domain and the target domain, the present invention selects one expert instead of adding the results of different experts by weights. The routing network is implemented using a simple CNN and introduces the domain label y i to balance the expert selection. For a 3D radar image input, the routing network will determine the source of the input and assign an expert to process the input data to obtain the corresponding high-resolution imaging result. The output can then be expressed as:

[0035] o = αW0x encoder + βΔWx encoder

[0036] where W0 are the parameters of the frozen branch, and ΔW are the parameters of the trainable branch. α and β are the classification results of the routing network and can be expressed as:

[0037]

[0038] where ROUND(·) is the rounding operation, C(·) is the linear layer of the binary classification operation, and W g are the trainable parameters of the routing network. The final output can be expressed as:

[0039] o = αW0x encoder+βΔWx encoder

[0040] = ROUND(C(xW g )) × [W0x encoder , ΔWx encoder T

[0041] After the above steps, the routing network can assign different experts to process the inputs from different domains. Since the parameters of the expert for restoring the target shape in the source domain are not updated, the knowledge of the source domain is retained. At the same time, the network learns new knowledge of the target domain.

[0042] Step 4: Input the BP radar image of the source domain or target domain sample into the target domain network to obtain a high-resolution imaging result containing shape and contour information.

[0043] Beneficial effects:

[0044] The network designed by the present invention can reduce the forgetting of source domain knowledge while learning new knowledge of the target domain, and at the same time has the ability to perform high-resolution imaging on the targets in the source domain and the target domain, which is an effective method for through-wall radar high-resolution imaging with less forgetting:

[0045] 1. Compared with the imaging result of the traditional imaging algorithm which is in the shape of a blurred light spot, the network designed by the present invention can achieve through-wall radar high-resolution imaging, and the imaging result contains the shape and category information of the imaged target, which can be directly identified and used.

[0046] 2. Compared with the deep learning method trained from scratch, the present invention adopts transfer learning. When the number of target domain samples is small, it can generate higher-quality images with fewer iterations, having clearer shapes and more realistic details, and being closer to the labels.

[0047] 3. Compared with the ordinary neural network which will forget the knowledge of the source domain after transfer learning and can only perform high-resolution imaging on the target domain input and no longer on the source domain input, the network designed by the present invention can alleviate the forgetting of source domain knowledge and at the same time has the ability to perform high-resolution imaging on the targets in the source domain and the target domain. Description of the drawings

[0048] Figure 1 : Explanation of the phenomenon that the network forgets the source domain knowledge after fine-tuning in the target domain;

[0049] Figure 2 : Flow chart of the implementation method;

[0050] Figure 3 : Signal scene schematic diagram;

[0051] ​Figure 4 1. The generator and discriminator structures for the optical-assisted high-resolution imaging network;

[0052] Figure 5 2. The schematic diagram of the target domain network LFHRNet structure;

[0053] Figure 6 3. The simulation dataset example, where (a) is the simulation model diagram, (b) is the dataset label, (c) is the 3D BP radar image, and (d) and (e) are the top view and front view respectively;

[0054] Figure 7 4. The schematic diagram of the measured scene, where (a) is the non-wall-penetrating scene, (b) is the radar behind the wall in the wall-penetrating scene, and (c) is the schematic diagram of the target placement in the wall-penetrating scene;

[0055] Figure 8 5. The comparison of the image generation results of LFHRNet and training from scratch with the change of training iteration times;

[0056] Figure 9 6. The comparison of the cosine similarity and PSNR of LFHRNet and training from scratch with the change of training iteration times at different numbers of training samples;

[0057] Figure 10 7. Some examples of the test results of LFHRNet on the simulation data in the target domain;

[0058] Figure 11 8. Some examples of the test results of LFHRNet on the measured data in the target domain;

[0059] Figure 12 9. Some examples of the test results of LFHRNet on the simulation and measured data in the source domain. Specific implementation mode

[0060] The present invention will be described in detail below with reference to the accompanying drawings and by way of examples.

[0061] The purpose of the present invention is to overcome the deficiencies of the existing traditional through-wall radar imaging algorithms in resolution, the difficulty of generalizing deep learning methods to new targets and the poor quality of generated images due to insufficient new target data volume, and the inability to perform high-resolution imaging of source domain targets after network fine-tuning, and to provide a through-wall radar high-resolution imaging method with less forgetting. Figure 2 This is the flowchart of the implementation mode of the present invention, and its specific steps include:

[0062] Step 1: Obtain the BP radar imaging results and the corresponding optical images, and construct the data:

[0063] Considering as Figure 3The imaging scene of size N containing a single reflection x ×N y is such that the radar transmitted signal s(t) is a stepped-frequency signal with a starting frequency of f0, a stepped-frequency interval of Δf, and contains K frequency points. Then the frequencies within the bandwidth are:

[0064] f k = f0 + kΔf, where k = 0,..., K - 1.

[0065] Assume the wall thickness is d wall , and the dielectric constant is ε r . Moving the antenna at an interval of d1, the m-th antenna position is (x m , -d1). Considering the direct reflection and single-reflection signals of the target, the direct reflection signal of the wall, and the reflection signal between the targets, the received echo signal at the m-th antenna position and the k-th frequency can be regarded as the superposition of the target reflection signal the direct reflection signal of the wall the reflection signal between the targets and noise v m,k , and can be expressed as:

[0066]

[0067] where P is the number of targets, W is the number of direct reflection paths of the wall. R is the number of multiple single-reflection paths on the target, σ p and σ pq are the reflection coefficients of the w-th direct reflection path of the wall, the p-th target, and the path between the p-th and q-th targets respectively. Taking Path-A shown in Figure 3 as an example, τ pm is the round-trip delay from the m-th antenna to the p-th target, is the round-trip delay of the w-th direct reflection path of the wall, and can be expressed as:

[0068]

[0069] where l1 is the propagation distance of the electromagnetic wave in the wall, l2 and l3 are the propagation distances of the electromagnetic wave in the air, is the length of the w-th direct reflection path of the wall, v is the propagation speed of the electromagnetic wave in the wall, and according to Snell's law, v can be expressed as:

[0070]

[0071] where c is the speed of light.

[0072] For three-dimensional BP imaging, the imaging region is divided into grids along the xyz axes, and the echo data of M groups of antennas at different positions are collected. The echo data is subjected to IFFT transformation to obtain s(m,k), and the two-way time delay τ between the m-th antenna position and the h-th pixel in the imaging region is calculated. mh Then, the imaging result of the h-th pixel is:

[0073]

[0074] Calculate the imaging of all pixels for the entire region.

[0075] Repeat the above steps, collect the echo data of different targets at different rotation angles and perform imaging to construct a complete simulation data set. And conduct actual measurement experiments to construct a complete actual measurement data set.

[0076] Step 2: Build and train a high-resolution imaging network assisted by source domain optical images:

[0077] The present invention uses Conditional Generative Adversarial Networks (cGAN) for high-resolution imaging. cGAN consists of a generator and a discriminator, as Figure 4 shown. The input of the network is a 3D radar image with a size of 32×32×32 and a 2D label with a size of 128×128.

[0078] The generator contains an encoder and a decoder. The encoder-decoder structure can extract abstract features from the input 3D radar image and use the decoder to reconstruct a high-quality 2D image from this compressed representation. The encoder consists of 5 sequentially connected 3D convolutional layers, which can map the 3D radar image into a 2048×1 vector. After each 3D convolutional layer, there is a BatchNorm layer and a LeakyReLU layer. The BatchNorm layer is for stabilizing the training and improving the convergence of the network, and the LeakyReLU layer introduces non-linearity to prevent gradient vanishing or descent. The decoder consists of 5 sequentially connected 2D transposed convolutional layers, BatchNorm layers, and ReLU layers, which upsample the vector to reconstruct an image of 128×128. At the same time, in order to retain high-frequency information, the network uses skip connections, splices the generated image output by the decoder with the high-frequency channels of the radar image, and then obtains the final output through 2 transposed convolutional layers.

[0079] The discriminator simultaneously inputs the 3D radar image and the 2D label or the generated image, and uses two different networks to transform both into vectors. The structure of the 3D input end is the same as that of the encoder of the generator, and the 2D input end consists of 8 sequentially connected 2D convolutional layers, BatchNorm layers, and LeakyReLU layers. The outputs of both are concatenated into a vector and classified through a Linear layer. The classification result D(x, y) or D(x, G(x)) represents the probability that the 2D input is a real label. During the training phase, the discriminator is used to distinguish the real label y and the generated image G(x), and the generator is used to generate an image that meets the conditional constraints, making it as indistinguishable by the discriminator as possible. This is a game process, and the loss function of this process can be expressed as:

[0080]

[0081] To make the generated image have a better effect, the present invention introduces the root mean square error (MSE) and perceptual loss when training the cGAN network. Then, the loss function of the generator during the training process is:

[0082] L(G) = L GAN (G) + λ MSE L MSE + λ p L p

[0083] Among them, L GAN (G) is the adversarial loss of the generative adversarial network itself, L MSE is the MSE loss between the real label and the generated image to ensure the accuracy and consistency of the generated image and the label at the pixel level. L p is the perceptual loss calculated through the pre-trained VGG network to make the generated image more in line with human eye perception. λ MSE and λ p are constants, making the generated image correct in details and directly recognizable by the human eye at the same time. During training, the generator is optimized to maximize the probability that the generated image is classified as a real label and minimize L MSE and L p , and the discriminator is optimized to minimize the classification error probability between the generated image and the real label.

[0084] Step 3: Build and train the target domain network LFHRNet:

[0085] The structure of the target domain network LFHRNet designed by the present invention is as Figure 5As shown, the target domain network has the same encoder structure as the source domain network and integrates two parallel branches as experts specialized in handling different tasks. The tasks in the source domain and the target domain respectively refer to high-resolution imaging of the source domain input and high-resolution imaging of the target domain input. When training the target domain network on target domain data, one branch is initialized with the source domain network and frozen during training to retain source domain knowledge, while the other expert's parameters are updated to learn new target domain knowledge. The training objective can be expressed as:

[0086]

[0087] where D t is the target domain dataset, containing 3D radar images x and 2D labels y. Θ encoder are the parameters pre-trained in the source domain, containing the knowledge learned in the source domain. The encoder of the target domain network is initialized with Θ encoder and frozen during training on D t as an off-the-shelf feature extractor. ΔW are the parameters learned on target domain data, and the optimal ΔW is obtained by minimizing the loss L t .

[0088] To integrate the two experts into one network, the present invention introduces a routing network to achieve the division of labor and precise allocation of the experts. When a 3D radar image is input into the target domain network, the 3D radar image may come from the source domain or the target domain. Due to the different data characteristics and distributions of the source domain and the target domain, the present invention selects one expert instead of adding the results of different experts by weights. The routing network is implemented using a simple CNN and introduces the domain label y i to balance the expert selection. For a 3D radar image input, the routing network will judge the source of the input and assign an expert to process the input data to obtain the corresponding high-resolution imaging result. The output can then be expressed as:

[0089] o = αW0x encoder + βΔWx encoder

[0090] where W0 are the parameters of the frozen branch, and ΔW are the parameters of the trainable branch. α and β are the classification results of the routing network and can be expressed as:

[0091]

[0092] where ROUND(·) is the rounding operation, C(·) is the linear layer of the binary classification operation, and W g are the trainable parameters of the routing network. The final output can be expressed as:

[0093] o = αW0x encoder+βΔWx encoder

[0094] =ROUND(C(xW g ))×[W0x encoder ,ΔWx encoder T

[0095] After the above steps, the routing network can assign different experts to process the inputs from different domains. Since the parameters of the expert for restoring the target shape of the source domain are not updated, the knowledge of the source domain is retained. At the same time, the network learns new knowledge of the target domain.

[0096] Step 4: Input the BP radar images of the source domain or target domain samples into the target domain network to obtain a high-resolution imaging result containing shape and contour information.

[0097] Thus, a high-resolution imaging method for through-wall radar with less forgetting is completed.

[0098] Embodiment

[0099] According to Step 1, set the simulated transmit signal as a stepped-frequency signal, with the antenna stepping 5 cm, the imaging area being 3 m × 3 m × 4 m, and construct a dataset containing tables, chairs, people, and RPGs with different rotation angles in the simulation scenario. The source domain dataset includes Chair-1 sim ), Chair-2 sim ), Table-1 sim ), and RPG, and the target domain dataset includes Table-2 sim ) and Human sim ). Set the measured transmit signal as a stepped-frequency signal, with a frequency interval of 2 MHz and a frequency range of 1.7 GHz - 2.2 GHz. The antenna array is 10 transmit and 10 receive, with a size of approximately 40 cm × 40 cm. In the non-through-wall scenario, the distance between the radar and the target is 2 m. In the through-wall scenario, the wall thickness is 0.2 m, the radar is close to one side of the wall, and the target is 2 m away from the other side of the wall. Collect a dataset containing tables, chairs, and people with different rotation angles. The source domain includes Chair-1 exp ) and Table (Table-1 exp ), and the target domain includes Chair-2 exp ) and Human exp ).

[0100] According to Step 2, set the batch size to 4, the optimizer as the Adam optimizer with an initial learning rate of 1e-2, and train the source domain cGAN network so that it can restore the shape of the source domain target and use it as a pre-trained network. ​

[0101] According to Step 3, some parameters of the target domain network LFHRNet are initialized using the cGAN network obtained in Step 2, and then the target domain network is trained using the target domain dataset. For comparison, the present invention also uses two other strategies, namely training a cGAN network from scratch using the target domain dataset, and fine-tuning on the target domain dataset after initializing the target domain using the cGAN network.

[0102] According to Step 4, the target domain network is used as the final high-resolution imaging network to process the input radar images of the source domain and the target domain. The size of the target domain training dataset is an important factor affecting the quality of the generated images. Therefore, the present invention analyzes the influence of the size of the target domain training dataset. There are 42, 84, 206, 330, and 412 samples in the target domain respectively, and both are trained from scratch on the same dataset for comparison.

[0103] Figure 8 For the generated images trained on 42 target domain samples for different rounds, compared with training from scratch, LFHRNet can generate higher-quality images with fewer iterations, having clearer shapes and more realistic details, and being closer to the labels. The fewer the number of iterations, the more obvious the advantage of LFHRNet in the quality of the generated images. As the number of iterations increases, the images generated by training from scratch gradually approach those generated by LFHRNet, but at the end of training, LFHRNet still has an advantage. Figure 9 It is a change curve graph of the cosine similarity and PSNR calculated on the training samples during the training process. As the number of iterations increases, the cosine similarity and PSNR of both training from scratch and LFHRNet gradually increase. As Figure 9 (a)(b) show, when the number of training samples is limited, the cosine similarity and PSNR of LFHRNet are higher than those of training from scratch, indicating that it is beneficial to initialize some layers of the target domain network and freeze them during training when the number of training samples is limited. As Figure 9 (c)(d) show, when the training data is relatively sufficient and the number of iterations is small, the generation result of LFHRNet is better than that of training from scratch, but when the number of iterations becomes larger, the generation results of both are similar.

[0104] Table 1 shows the results of cosine similarity, PSNR, LPIPS, and SSIM on the target domain test set with different sizes of the target domain training dataset. When the number of training samples is limited, LFHRNet is superior to training from scratch. When the training data is relatively sufficient, the results of training from scratch are better than LFHRNet. Regardless of the number of training samples, fine-tuning has the best effect among the three. This is because fine-tuning initializes and updates all layers, can transfer the knowledge of the source domain, and makes the training more sufficient. Although LFHRNet also utilizes the knowledge of the source domain, it freezes some layers during training. And there is a situation where the expert assignment of the routing network is incorrect, and the input of the target domain may be reconstructed into a shape closer to the target of the source domain. Although the performance of LFHRNet on the target domain is slightly inferior to fine-tuning, it can meet the requirements for achieving high-resolution imaging. Figure 10 and Figure 11 Some examples of the test results of the simulated target domain and the measured target domain are shown. It can be seen that LFHRNet can generate high-resolution imaging results containing shape, contour, and texture information. And the main advantage of LFHRNet lies in the source domain.

[0105] Statistical comparison of the results on the target domain test set

[0106]

[0107]

[0108] Although fine-tuning improves the imaging results of the target domain samples, it will forget the knowledge learned in the source domain. To verify the effectiveness of LFHRNet in preventing the forgetting of source domain knowledge, the present invention tests it on the source domain. Figure 12Shows the test results on the simulated and measured data in the source domain. Column (a) shows the schematic diagrams of the targets in the simulated and measured scenarios, column (b) shows the labels, column (c) shows the 3D radar images, columns (d) and (e) show the top view and front view of the 3D radar images respectively, and columns (f) to (h) show the generation results of training from scratch, fine-tuning, and LFHRNet. It can be seen that the network fine-tuned in the target domain forgets the knowledge learned in the source domain and cannot correctly reconstruct the shape of the targets in the source domain, and the generation results become as poor as those of training from scratch, while training from scratch has never been trained on the source domain data. This is because all parameters are updated during fine-tuning in the target domain, destroying the knowledge originally stored in the network. However, LFHRNet can correctly generate high-resolution images close to the labels, indicating that LFHRNet is effective in avoiding forgetting source domain knowledge. This is because when training LFHRNet in the target domain, one expert is frozen to retain the source domain knowledge. Considering that LFHRNet can also perform high-resolution imaging on the target domain samples, it can be seen that the two experts of LFHRNet have obtained the ability to handle tasks in different domains, and the routing network can allocate appropriate experts for different inputs. Table 2 shows the results of cosine similarity, PSNR, LPIPS, and SSIM on the source domain test set when the size of the target domain training dataset is different. It can be seen that LFHRNet has significant advantages. And different sizes of the target domain training dataset have little impact on the performance of LFHRNet in the source domain.

[0109] Table 2 Comparative Statistics of Results on the Source Domain Test Set

[0110]

[0111] In summary, the present invention proposes a high-resolution imaging method for through-wall radar with less forgetting. LFHRNet adopts a multi-branch structure. One branch is updated during training to learn new knowledge on the target domain samples, and one branch is frozen to retain the source domain knowledge. Simulation and experimental results prove that the proposed method can effectively alleviate the problem of the network forgetting source domain knowledge after fine-tuning in the target domain and can simultaneously have the ability to perform high-resolution imaging on the targets in both the source domain and the target domain.

[0112] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of the present invention, several improvements and deformations can still be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.

Claims

1. A less forgotten through-wall radar high-resolution imaging method, characterized in that: include: Step 1: Obtain BP radar imaging results and corresponding optical images to construct a data set; Step 2: Build and train a high-resolution imaging network assisted by source domain optical images; Step 3: Build and train the target domain network LFHRNet; Step 4: Input the BP radar images of the target domain and source domain samples into the target domain network to obtain high-resolution imaging results containing shape and contour information.

2. A method for high-resolution through-wall radar imaging with little forgetfulness as claimed in claim 1, characterized in that: In step 1, for a size of N including one reflection x ×N y In the imaging scenario, the radar transmission signal s(t) is a stepped frequency signal with a starting frequency of f0, a stepped frequency interval of Δf, and K frequency points. The frequency within the bandwidth is: f k =f0+kΔf,k=0,...,K-1.。 3. A through-wall radar high-resolution imaging method with little forgetting as claimed in claim 1, characterized in that: In step 1, let the wall thickness be d wall , the dielectric constant is ε r , move the antenna by interval d1, then the position of the mth antenna is (x m ,-d1); Considering the direct reflection and primary reflection signals of the target, the direct reflection signal of the wall and the reflection signal between the targets, the received echo signal at the mth antenna position and the kth frequency can be regarded as the target reflection signal The wall directly reflects the signal Reflected signals between targets and noise v m,k The superposition can be expressed as: Where P is the number of targets, W is the number of direct reflection paths from the wall, and R is the number of multiple one-time reflection paths on the target. σ p and σ pq are the reflection coefficients of the wth wall direct reflection path, the pth target, and the path between the pth and qth targets, respectively.

4. A method for high-resolution through-wall radar imaging with little forgetfulness as claimed in claim 1, characterized in that: In step 1, τ pm is the round-trip delay from the mth antenna to the pth target, is the round-trip delay of the w-th wall direct reflection path, expressed as: Among them, l1 is the propagation distance of electromagnetic waves in the wall, l2 and l3 are the propagation distances of electromagnetic waves in the air, is the length of the direct reflection path of the wth wall, v is the propagation speed of the electromagnetic wave in the wall, according to Snell's law, v can be expressed as where c is the speed of light.

5. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 1, for 3D BP imaging, the imaging area is divided into grids along the xyz axis, and the echo data of M groups of antennas at different positions are collected. The echo data are transformed by IFFT to obtain s(m,k), and the two-way delay τ between the mth antenna position and the hth pixel in the imaging area is calculated. mh , then the imaging result of the h-th pixel is 6. The method for high-resolution through-wall radar imaging with little forgetfulness as claimed in claim 1, characterized in that: In step 2, a conditional generative adversarial network (cGAN) is used for high-resolution imaging. The cGAN consists of a generator and a discriminator. The input of the network is a 3D radar image of size 32×32×32 and a 2D label of size 128×128.

7. The method for high-resolution through-wall radar imaging with little forgetfulness as claimed in claim 1, characterized in that: In step 2, the generator contains an encoder and a decoder. The encoder-decoder structure can extract abstract features from the input 3D radar image and use the decoder to reconstruct a high-quality 2D image from the compressed representation; the encoder consists of 5 sequentially connected 3D convolutional layers, which can map the 3D radar image into a 2048×1 vector; each 3D convolutional layer is followed by a BatchNorm layer and a LeakyReLU layer; the decoder consists of 5 sequentially connected 2D transposed convolutional layers, a BatchNorm layer, and a ReLU layer, which upsamples the vector to reconstruct a 128×128 image.

8. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 2, the network uses skip connections to concatenate the generated image output by the decoder with the high-frequency channel of the radar image, and then obtains the final output through two transposed convolutional layers.

9. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 2, the discriminator simultaneously inputs the 3D radar image and the 2D label or generated image, and uses two different networks to transform the two into vectors. The structure of the 3D input end is the same as the encoder structure of the generator. The 2D input end consists of 8 2D convolutional layers, BatchNorm layers, and LeakyReLU layers connected in sequence. The outputs of the two are concatenated into a vector and classified through the Linear layer. The classification result D(x,y) or D(x,G(x)) represents the probability that the 2D input is the true label.

10. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 2, during the training phase, the discriminator is used to distinguish between the true label y and the generated image G(x), and the generator is used to generate images that meet the conditional constraints and make it as difficult for the discriminator to distinguish as possible; the loss function of this process can be expressed as: When training the cGAN network, the mean square error (MSE) and perceptual loss are introduced, and the generator loss function during training is: L(G)=L GAN (G)+λ MSE L MSE +λ p L p Among them, L GAN (G) is the adversarial loss of the generative adversarial network itself, L MSE is the MSE loss between the true label and the generated image, and L p is the perceptual loss calculated by the pre-trained VGG network, and is λ MSE and λ p is a constant.

11. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 2, during training, the generator is optimized to maximize the probability that the generated image is classified as the true label and minimize L MSE and L p , the discriminator is optimized to minimize the probability of misclassification of generated images and true labels.

12. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 3, a target domain network LFHRNet is designed, which has the same encoder structure as the source domain network and integrates two parallel branches as experts specializing in different tasks. The tasks in the source domain and the target domain refer to high-resolution imaging of the source domain input and high-resolution imaging of the target domain input, respectively. When the target domain network is trained on the target domain data, one of the branches is initialized with the source domain network and frozen during training to retain the source domain knowledge, and the other expert parameters are updated to learn new knowledge in the target domain. The training goal can be expressed as: Among them, D t is the target domain dataset, containing 3D radar images x and 2D labels y, Θ encoder is the parameter pre-trained in the source domain, which contains the knowledge learned in the source domain. The encoder of the target domain network uses Θ encoder Initialize and t Frozen during training, as a ready-made feature extractor, ΔW is the parameter learned on the target domain data, by minimizing the loss L t Get the optimal ΔW.

13. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 3, a routing network is introduced to realize the division of labor and precise allocation of experts. When the 3D radar image is input into the target domain network, the 3D radar image may come from the source domain or the target domain. Due to the different data characteristics and distributions of the source domain and the target domain, the present invention selects one expert instead of adding the results of different experts according to the weights. The routing network is implemented using a simple CNN, and the domain label y is introduced. i To balance the expert selection.

14. The method for high-resolution through-wall radar imaging with little forgetting as claimed in claim 1, characterized in that: In step 3, for a 3D radar image input, the routing network will determine the source of the input and assign an expert to process the input data to obtain the corresponding high-resolution imaging result; the output can be expressed as: o=αW0x encoder +βΔWx encoder Among them, W0 is the parameter of the frozen branch, ΔW is the parameter of the trainable branch, α and β are the classification results of the routing network, expressed as: Among them, ROUND(·) is the rounding operation, C(·) is the linear layer of the binary classification operation, and W g is the trainable parameter of the routing network; the final output is expressed as

Images