Learning assistance-based ISAR image super-resolution method and device and computer equipment
By constructing an ISAR image super-resolution network using a learning-assisted method, and combining iterative computation and deep neural network updates, the problems of insufficient interpretability and generalization performance in ISAR image super-resolution are solved, generating high-quality, high-resolution ISAR images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NAT UNIV OF DEFENSE TECH
- Filing Date
- 2022-10-13
- Publication Date
- 2026-07-31
AI Technical Summary
Existing ISAR image super-resolution methods suffer from insufficient interpretability and generalization performance when generating high-resolution images. In particular, model-based methods have limited quality, while deep learning-based methods lack good interpretability.
A learning-assisted approach is adopted to construct an ISAR image super-resolution network by generating an initial ISAR image and a degenerate blur kernel. The network is then updated using an iterative computation framework and a deep neural network, combined with reconstruction error and blur kernel-assisted updates, until the image and blur kernel converge, thus generating a high-resolution ISAR image.
While ensuring the performance of the ISAR image super-resolution algorithm, it also has good interpretability and generalization performance, and can generate high-quality high-resolution ISAR images by utilizing known prior information.
Smart Images
Figure CN115511713B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of ISAR image processing technology, and in particular to a learning-assisted ISAR image super-resolution method, apparatus and computer device. Background Technology
[0002] Super-resolution methods for inverse synthetic aperture radar (ISAR) images play a crucial role in ISAR target detection, identification, and classification tasks.
[0003] However, in practical applications, the acquired ISAR images often have low resolution, which significantly affects the efficiency of subsequent ISAR target detection and recognition. Therefore, super-resolution methods for generating high-resolution ISAR images from low-resolution ISAR images urgently need to be studied.
[0004] Currently, existing learning-based ISAR image super-resolution methods are mainly divided into two categories: model-based ISAR image super-resolution methods and deep learning-based ISAR image super-resolution methods. Model-based ISAR image super-resolution methods, based on mathematical foundations, possess good interpretability and generalization performance, and can utilize known prior information. However, their drawback is that the quality of the generated super-resolution ISAR images is very limited. Deep learning-based methods have been widely researched and applied in ISAR image super-resolution. These algorithms exhibit excellent data-driven performance, but their disadvantages include a lack of good interpretability and generalization performance. Summary of the Invention
[0005] Therefore, it is necessary to provide a learning-assisted ISAR image super-resolution method, apparatus, and computer device with good interpretability and generalization performance to address the above-mentioned technical problems.
[0006] A learning-assisted ISAR image super-resolution method, the method comprising:
[0007] Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size;
[0008] Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel.
[0009] An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0010] During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0011] After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0012] In one embodiment, the initial ISAR image is generated using random white Gaussian noise.
[0013] In one embodiment, the reconstruction error is calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degraded blur kernel output in each iteration, including:
[0014] The intermediate ISAR image is convolved with the degenerate blur kernel and downsampled to obtain an intermediate ISAR low-resolution image corresponding to the intermediate ISAR image, wherein the downsampling operation is performed according to the super-resolution scaling factor.
[0015] The reconstruction error is calculated based on the intermediate ISAR image, the intermediate low-resolution ISAR image, and the low-resolution ISAR image.
[0016] In one embodiment, updating the ISAR image super-resolution network based on the reconstruction error includes:
[0017] Gradient calculation is performed based on the reconstruction error, and the parameters in the ISAR image super-resolution network are iteratively updated based on the calculation results.
[0018] In one embodiment, updating the degenerate fuzzy kernel based on the reconstruction error and the fuzzy kernel-assisted update network includes:
[0019] The gradient of the degenerate fuzzy kernel is calculated based on the reconstruction error, and the calculation result is input into the fuzzy kernel auxiliary update network to obtain the fuzzy kernel update value;
[0020] The degenerate fuzzy kernel is updated based on the fuzzy kernel update value.
[0021] In one embodiment, after updating the degenerate fuzzy kernel according to the fuzzy kernel update value, the parameters in the fuzzy kernel-assisted update network are further corrected according to the updated degenerate fuzzy kernel, including:
[0022] The reconstruction error is recalculated based on the updated degenerate fuzzy kernel. Gradients are calculated based on this reconstruction error, and the parameters in the fuzzy kernel-assisted update network are corrected in reverse based on the calculation results for iterative updates.
[0023] In one embodiment, during the iterative training of the ISAR image super-resolution network, the ISAR image super-resolution network and the blur kernel-assisted update network are repeatedly iterated alternately until the high-resolution ISAR image and the degenerate blur kernel both converge.
[0024] In one embodiment, when the ISAR image super-resolution network and the fuzzy kernel-assisted update network are repeatedly iterated alternately, the ISAR image super-resolution network is iterated once, and the fuzzy kernel-assisted update network is iterated 10 times.
[0025] A learning-assisted ISAR image super-resolution device, the device comprising:
[0026] The relevant parameter acquisition module is used to acquire the low-resolution ISAR image to be super-resolution and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degeneracy blur kernel size.
[0027] The initial ISAR image and degraded blur kernel generation module is used to generate initial ISAR images using random noise according to the size of the high-resolution image, and to generate degraded blur kernels from the mean according to the size of the degraded blur kernel;
[0028] The high-resolution ISAR image acquisition module is used to construct an ISAR image super-resolution network based on a learning-assisted update and iterative computing framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining the high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0029] The ISAR image super-resolution network update module is used to update the ISAR image super-resolution network during the iteration of the initial ISAR image based on the reconstruction error calculated by the intermediate ISAR image, the low-resolution ISAR image and the degenerate blur kernel output in each iteration, and then input the intermediate ISAR image into the updated ISAR image super-resolution network to continue the iteration.
[0030] The degenerate blur kernel update module is used to update the degenerate blur kernel according to the reconstruction error and the blur kernel-assisted update network after each update of the ISAR image super-resolution network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0031] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:
[0032] Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size;
[0033] Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel.
[0034] An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0035] During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0036] After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0037] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0038] Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size;
[0039] Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel.
[0040] An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0041] During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0042] After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0043] The aforementioned learning-assisted ISAR image super-resolution method, apparatus, and computer device generate an initial ISAR image and a degenerate blur kernel using random noise based on the size of the high-resolution image and the size of the degenerate blur kernel, respectively. The initial ISAR image is then input into an ISAR image super-resolution network constructed using a learning-assisted update iterative computation framework, and it is iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image. During this process, the ISAR image super-resolution network is updated based on the reconstruction error calculated from the intermediate ISAR image, low-resolution ISAR image, and degenerate blur kernel output in each iteration. The intermediate ISAR image is then input into the updated ISAR image super-resolution network for further iteration. After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel to assist in updating the network. The updated degenerate blur kernel is used to calculate a new reconstruction error in the next iteration. This method, through learning assistance, introduces a deep neural network into the original model-based alternating projection minimization computation framework. While ensuring the good performance of the ISAR image super-resolution algorithm, it also has good interpretability and generalization performance. Furthermore, it can utilize known prior information and achieve good performance even with prior information. Attached Figure Description
[0044] Figure 1 This is a flowchart illustrating a learning-assisted ISAR image super-resolution method in one embodiment.
[0045] Figure 2 This is a schematic diagram of the U-Net network structure in one embodiment;
[0046] Figure 3 This is a schematic diagram of the structure of the ISAR image super-resolution network and the blur kernel-assisted update network in one embodiment;
[0047] Figure 4 This is a schematic diagram showing the performance test results of different fuzzy kernels in the experiment;
[0048] Figure 5 This is a schematic diagram of the intermediate results of the ISAR image super-resolution network iteration in the experiment;
[0049] Figure 6 This is a schematic diagram of the final results of the ISAR image super-resolution method iteration in the experiment;
[0050] Figure 7 This is a structural block diagram of a learning-assisted ISAR image super-resolution device in one embodiment;
[0051] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0053] like Figure 1 As shown, a learning-assisted ISAR image super-resolution method is provided, including the following steps:
[0054] Step S100: Obtain the low-resolution ISAR image to be super-resolution and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degeneracy blur kernel size.
[0055] Step S110: Generate an initial ISAR image and a degraded blur kernel using random noise based on the size of the high-resolution image and the size of the degraded blur kernel.
[0056] Step S120: Construct an ISAR image super-resolution network based on a learning-assisted update and iterative computing framework. Input the initial ISAR image into the ISAR image super-resolution network and iterate it until the output ISAR image converges. This yields a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0057] Step S130: During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0058] In step S140, after each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0059] It should be noted that the above method steps are not performed sequentially according to the numbering order. In fact, steps S130 and S140 are both part of step S120.
[0060] In this embodiment, the learning-assisted ISAR image super-resolution method introduces a deep learning network to iteratively solve the blur kernel in ISAR image super-resolution. This method not only ensures effectiveness and practicality, but also has advantages such as strong interpretability and good generalization ability.
[0061] In step S100, the low-resolution ISAR image to be super-resolution analyzed is acquired as a known image, i.e., the low-resolution image to be solved. There are no strict restrictions on the settings of the super-resolution related parameters; in principle, common sizes can be used for subsequent solutions. These related parameters are only used for initializing subsequent network parameters. The super-resolution scaling factor is set according to the size of the low-resolution ISAR image to be super-resolution analyzed.
[0062] In step S110, the initial ISAR image is generated using random white Gaussian noise. The initial degenerate blur kernel is generated using the mean.
[0063] In this embodiment, the initial ISAR image and the size of the degraded blur kernel are consistent with the parameters of the low-resolution ISAR image and the degraded blur kernel.
[0064] In this embodiment, the parameters of the ISAR image super-resolution network are also initialized based on the high-resolution image size and the degenerate blur kernel size.
[0065] Step S120 actually includes steps S130 and S140, and these two steps are performed alternately. First, the initial ISAR image is iterated using an ISAR image super-resolution network. After each iteration, the network is updated, and the degenerate blur kernel is also updated after each update. Essentially, steps S130 and S140 are implemented alternately until the ISAR image output by the ISAR image super-resolution network and the degenerate blur kernel both converge. At this point, the ISAR image output by the ISAR image super-resolution network is the high-resolution ISAR image obtained after super-resolution of the low-resolution ISAR image, which is the solution obtained.
[0066] In this embodiment, the ISAR image super-resolution network adopts the U-Net network, and the input and output sizes are consistent with the sizes of known low-resolution ISAR images. The structure of the U-Net network is as follows: Figure 2 As shown.
[0067] In step S130, the reconstruction error is calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration. This includes: convolving the intermediate ISAR image with the degenerate blur kernel and performing downsampling operations to obtain an intermediate ISAR low-resolution image corresponding to the intermediate ISAR image, wherein the downsampling operation is performed according to the super-resolution scaling factor. The reconstruction error is then calculated based on the intermediate ISAR image, the intermediate ISAR low-resolution image, and the low-resolution ISAR image.
[0068] Specifically, such as Figure 3 As shown, the ISAR image super-resolution network is defined as G, and θ is used as the network name.x The parameters of this network are represented by z, which is an initial ISAR image randomly generated with noise. x Then the ISAR image super-resolution network uses parameter θ x The process of generating the intermediate ISAR image x can be represented as x = G(z) x ,θ x In fact, the intermediate ISAR image x is a high-resolution image relative to the input image. Then, the generated intermediate ISAR image and the blur kernel k are used to calculate the ISAR image reconstruction error.
[0069] In fact, sequentially convolving and downsampling the intermediate ISAR image x with a blur kernel is the process of degrading the high-resolution intermediate ISAR image x into a low-resolution image. That is, generating a low-resolution image related to itself based on the intermediate ISAR image x, and finally using the sparsity characteristics of the ISAR image as prior information of the ISAR image to add to the calculation formula. The calculation process of the ISAR image reconstruction error is shown in the following formula:
[0070]
[0071] In formula (1), x represents the intermediate ISAR image, which is the image output by the ISAR image super-resolution network, and (x*k) represents the convolution operation between the intermediate ISAR image x and the blur kernel k. s This indicates a downsampling operation on the image after convolution, where s represents the super-resolution scaling factor, (x*k)↓ s y represents an intermediate low-resolution image, and y represents a known low-resolution ISAR image.
[0072] Next, updating the ISAR image super-resolution network based on the reconstruction error includes: calculating the gradient based on the reconstruction error, and iteratively updating the parameters in the ISAR image super-resolution network based on the calculation results.
[0073] Specifically, the ISAR image super-resolution network update process is shown in the following formula:
[0074]
[0075] In formula (2), the network update adopts the Adam update strategy with an update step size of γ. x .
[0076] In step S140, updating the degenerate fuzzy kernel based on the reconstruction error and the fuzzy kernel-assisted update network includes: first, calculating the gradient of the degenerate fuzzy kernel based on the reconstruction error, inputting the calculation result into the fuzzy kernel-assisted update network to obtain the fuzzy kernel update value, and then updating the degenerate fuzzy kernel based on the fuzzy kernel update value.
[0077] Specifically, the process of calculating the gradient of the degenerate fuzzy kernel k based on the reconstruction error is shown in the following equation:
[0078]
[0079] Specifically, the fuzzy kernel-assisted update network is defined as FCN, using θ k The parameters of the network are represented by θ, and the fuzzy kernel-assisted update network updates the network through the parameter θ. k The process of generating the fuzzy kernel update value is shown in the following formula:
[0080]
[0081] Specifically, the degenerate fuzzy kernel is updated based on the fuzzy kernel update value as shown in the following formula:
[0082] k←k+Δk (5)
[0083] The updated degenerate fuzz kernel is used to calculate the reconstruction error in the next iteration.
[0084] In this embodiment, after updating the degenerate fuzzy kernel, the parameters in the fuzzy kernel-assisted update network are also corrected based on the updated degenerate fuzzy kernel. This includes: recalculating the reconstruction error based on the updated degenerate fuzzy kernel, calculating the gradient based on the reconstruction error, and correcting the parameters in the fuzzy kernel-assisted update network in reverse based on the calculation results to iteratively update the network.
[0085] Specifically, the process of updating the fudged kernel update network is shown in the following formula:
[0086]
[0087] In this embodiment, the fuzzy kernel-assisted update network is a 3-layer fully connected network with input and output sizes consistent with the fuzzy kernel size.
[0088] In this embodiment, during the iterative training of the ISAR image super-resolution network, the ISAR image super-resolution network and the blur kernel-assisted update network are repeatedly iterated alternately until both the high-resolution ISAR image and the degenerate blur kernel converge. Essentially, this means continuously repeating and alternating steps S130 and S140 until the ISAR image output by the ISAR image super-resolution network meets the preset criteria, i.e., a high-resolution ISAR image is finally output. Simultaneously, the degenerate blur kernel, after multiple iterations, will also converge.
[0089] In this embodiment, when the ISAR image super-resolution network and the fuzzy kernel-assisted update network are iterated alternately, the ISAR image super-resolution network is iterated once, and the fuzzy kernel-assisted update network is iterated 10 times.
[0090] In some embodiments, step S130 requires a total of 1000 iterations. For each execution of step S130, step S140 is executed 10 times, and the output ISAR image and degenerate blur kernel are updated repeatedly in an alternating manner until convergence.
[0091] In this embodiment, the ISAR image super-resolution network and the fuzzy kernel-assisted update network in this method do not require pre-training or additional parameter tuning.
[0092] As can be seen from the above method, this method does not require training the network with samples, but directly processes the low-resolution ISAR image to be super-resolution to obtain the associated high-resolution ISAR image.
[0093] To better understand this method, we will use the following methods... Figure 3 The technical details of this method will be described and further explained. Figure 3 In the diagram, DIP represents the ISAR image super-resolution module, LPGD represents the fuzzy kernel update module, and after further expansion of DIP, G represents the ISAR image super-resolution network, RE represents the reconstruction error loss of the ISAR low-resolution image, FCN represents the fuzzy kernel auxiliary update network, SR represents the final ISAR super-resolution result, Result represents the fuzzy kernel prediction result, and GT represents the true value of the fuzzy kernel.
[0094] Furthermore, the ISAR image super-resolution network and the fuzzy kernel-assisted update network in this method can be trained in advance using a large number of ISAR sample images until the output ISAR image and the degenerate fuzzy kernel converge to obtain a trained ISAR image super-resolution network. In practical applications, it is only necessary to input the low-resolution ISAR image to be super-resolution into the pre-trained ISAR image super-resolution network to output a high-resolution ISAR image.
[0095] In this paper, the effectiveness of the learning-assisted ISAR image super-resolution method (hereinafter referred to as the method) proposed in this paper is also verified through experiments, including experiments with different blur kernel sizes and different image super-resolution scaling sizes.
[0096] Experiment content:
[0097] 1.1) First, the network performance of this method under different fuzzy kernel conditions was tested. The experimental results are as follows: Figure 4 As shown in the figure, the performance of our method (LPGD) is significantly better than other model-based methods (PAM) and deep learning-based methods (Double-DIP and FKP-DIP). The test results are quantitatively compared using peak signal-to-noise ratio (PSNR), and our method shows the best performance in the experimental results.
[0098] 1.2) Then, comparative experiments are conducted to expand the iterative processes of these methods, such as... Figure 5 As shown in the figure, in the early stages of iteration, the output of this method did not converge, resulting in poor visualization performance and a significant difference from the high-resolution ISAR image to be generated. From the 400th iteration (the fifth figure from the left), the performance of this method began to resemble an ISAR image, and the performance continued to steadily improve. The intermediate visualization results of this method's iterations are shown in the figure below. Figure 5 As shown. The final visualization result is as follows. Figure 6 As shown.
[0099] Experimental results show that this method, through learning assistance, introduces a deep neural network into the original model-based alternating projection minimization computation framework. This ensures that the ISAR image super-resolution algorithm has good performance, while also having good interpretability and generalization performance. Furthermore, it can utilize known prior information and achieve good performance even with prior information available.
[0100] In the aforementioned learning-assisted ISAR image super-resolution method, an initial ISAR image and a degenerate blur kernel are generated using random noise based on the size of the high-resolution image and the size of the degenerate blur kernel, respectively. The initial ISAR image is then input into an ISAR image super-resolution network constructed using a learning-assisted update iterative computation framework, and it is iterated until the output ISAR image converges, thus obtaining the high-resolution ISAR image. During this process, the ISAR image super-resolution network is updated based on the reconstruction error calculated from the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration. The intermediate ISAR image is then input into the updated ISAR image super-resolution network for further iteration. After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel to assist in updating the network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration. This method, through learning assistance, introduces a deep neural network into the original model-based alternating projection minimization computation framework. While ensuring the good performance of the ISAR image super-resolution algorithm, it also has good interpretability and generalization performance. Furthermore, it can utilize known prior information and achieve good performance even with prior information.
[0101] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.
[0102] In one embodiment, such as Figure 7 As shown, a learning-assisted ISAR image super-resolution device is provided, comprising: a correlation parameter acquisition module 200, an initial ISAR image and degenerate blur kernel generation module 210, a high-resolution ISAR image acquisition module 220, an ISAR image super-resolution network update module 230, and a degenerate blur kernel update module 240, wherein:
[0103] The relevant parameter acquisition module 200 is used to acquire the low-resolution ISAR image to be super-resolution and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor and the degeneracy blur kernel size.
[0104] The initial ISAR image and degraded blur kernel generation module 210 is used to generate an initial ISAR image based on the size of the high-resolution image using random noise, and to generate a degraded blur kernel based on the mean value according to the size of the degraded blur kernel;
[0105] The high-resolution ISAR image acquisition module 220 is used to construct an ISAR image super-resolution network based on a learning-assisted update and iterative computing framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining the high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0106] The ISAR image super-resolution network update module 230 is used to update the ISAR image super-resolution network during the iteration of the initial ISAR image based on the reconstruction error calculated by the intermediate ISAR image, the low-resolution ISAR image and the degenerate blur kernel output in each iteration, and then input the intermediate ISAR image into the updated ISAR image super-resolution network to continue the iteration.
[0107] The degenerate blur kernel update module 240 is used to update the degenerate blur kernel according to the reconstruction error and the blur kernel-assisted update network after each update of the ISAR image super-resolution network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0108] Specific limitations regarding the learning-assisted ISAR image super-resolution device can be found in the limitations of the learning-assisted ISAR image super-resolution method described above, and will not be repeated here. Each module in the aforementioned learning-assisted ISAR image super-resolution device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0109] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a learning-assisted ISAR image super-resolution method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0110] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0111] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0112] Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size;
[0113] Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel.
[0114] An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0115] During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0116] After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0117] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0118] Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size;
[0119] Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel.
[0120] An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image.
[0121] During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration.
[0122] After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration.
[0123] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0124] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A learning-assisted based ISAR image super-resolution method, characterized in that, The method includes: Acquire the low-resolution ISAR image to be super-resolution processed and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degenerate blur kernel size; Initial ISAR images are generated using random noise based on the size of the high-resolution image, and a degraded blur kernel is generated from the mean based on the size of the degraded blur kernel. An ISAR image super-resolution network is constructed based on a learning-assisted update and iterative computation framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining a high-resolution ISAR image after super-resolution of the low-resolution ISAR image. During the iteration of the initial ISAR image, the reconstruction error calculated based on the intermediate ISAR image, the low-resolution ISAR image, and the degenerate blur kernel output in each iteration is used to update the ISAR image super-resolution network. Then, the intermediate ISAR image is input into the updated ISAR image super-resolution network to continue the iteration. After each update of the ISAR image super-resolution network, the degenerate blur kernel is also updated based on the reconstruction error and the blur kernel-assisted update network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration. The reconstruction error is calculated based on the intermediate ISAR image, low-resolution ISAR image, and the degraded blur kernel output in each iteration, including: The intermediate ISAR image is convolved with the degenerate blur kernel and downsampled to obtain an intermediate ISAR low-resolution image corresponding to the intermediate ISAR image, wherein the downsampling operation is performed according to the super-resolution scaling factor. The reconstruction error is calculated based on the intermediate ISAR image, the intermediate low-resolution ISAR image, and the low-resolution ISAR image. The step of updating the degenerate fuzzy kernel based on the reconstruction error and the fuzzy kernel-assisted update network includes: The gradient of the degenerate fuzzy kernel is calculated based on the reconstruction error, and the calculation result is input into the fuzzy kernel auxiliary update network to obtain the fuzzy kernel update value; The degenerate fuzzy kernel is updated based on the fuzzy kernel update value; During the iterative training of the ISAR image super-resolution network, the ISAR image super-resolution network and the blur kernel-assisted update network are repeatedly iterated alternately until the high-resolution ISAR image and the degenerate blur kernel converge.
2. The ISAR image super-resolution method of claim 1, wherein, The initial ISAR image was generated using random Gaussian white noise.
3. The ISAR image super-resolution method of claim 1, wherein, Updating the ISAR image super-resolution network based on the reconstruction error includes: Gradient calculation is performed based on the reconstruction error, and the parameters in the ISAR image super-resolution network are iteratively updated based on the calculation results.
4. The ISAR image super-resolution method of claim 1, wherein, After updating the degenerate fuzzy kernel according to the fuzzy kernel update value, the parameters in the fuzzy kernel-assisted update network are also corrected according to the updated degenerate fuzzy kernel, including: The reconstruction error is recalculated based on the updated degenerate fuzzy kernel. Gradients are calculated based on this reconstruction error, and the parameters in the fuzzy kernel-assisted update network are corrected in reverse based on the calculation results for iterative updates.
5. The method of claim 1, wherein, When the ISAR image super-resolution network and the fuzzy kernel-assisted update network are iterated repeatedly, the ISAR image super-resolution network is iterated once, and the fuzzy kernel-assisted update network is iterated 10 times.
6. A learning-assisted ISAR image super-resolution device, used to implement the learning-assisted ISAR image super-resolution method according to any one of claims 1 to 5, characterized in that, The device includes: The relevant parameter acquisition module is used to acquire the low-resolution ISAR image to be super-resolution and the parameters related to super-resolution, including the high-resolution image size, the super-resolution scaling factor, and the degeneracy blur kernel size. The initial ISAR image and degraded blur kernel generation module is used to generate initial ISAR images using random noise according to the size of the high-resolution image, and to generate degraded blur kernels from the mean according to the size of the degraded blur kernel; The high-resolution ISAR image acquisition module is used to construct an ISAR image super-resolution network based on a learning-assisted update and iterative computing framework. The initial ISAR image is input into the ISAR image super-resolution network and iterated until the output ISAR image converges, thus obtaining the high-resolution ISAR image after super-resolution of the low-resolution ISAR image. The ISAR image super-resolution network update module is used to update the ISAR image super-resolution network during the iteration of the initial ISAR image based on the reconstruction error calculated by the intermediate ISAR image, the low-resolution ISAR image and the degenerate blur kernel output in each iteration, and then input the intermediate ISAR image into the updated ISAR image super-resolution network to continue the iteration. The degenerate blur kernel update module is used to update the degenerate blur kernel according to the reconstruction error and the blur kernel-assisted update network after each update of the ISAR image super-resolution network. The updated degenerate blur kernel is used to calculate the new reconstruction error in the next iteration. 7.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-6 when the computer program is executed by the processor. When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.