Integrated processing method of super-resolution and digital decoding for low-resolution intermediate coded images

Abstract

This invention discloses a method for super-resolution and digital decoding integration processing of low-resolution intermediate-coded images, comprising: establishing an image database including a high-resolution clear image set, a high-resolution intermediate-coded image set, and a low-resolution intermediate-coded image set; constructing a super-resolution sub-network and pre-training it using the low-resolution and high-resolution intermediate-coded image sets; constructing a super-resolution and digital decoding integrated network model, the super-resolution and digital decoding integrated network model including a cascaded super-resolution sub-network and a digital decoding sub-network; training the super-resolution and digital decoding integrated network model using the low-resolution intermediate-coded image set and the high-resolution clear image set; and inputting the original low-resolution intermediate-coded image into the trained super-resolution and digital decoding sub-network integrated network model to obtain the corresponding super-resolution decoded image. The method of this invention can effectively process blurry low-resolution intermediate-coded images into clear super-resolution decoded images.

Description

Technical Field

[0001] This invention belongs to the field of computational optical imaging technology, specifically relating to a method for super-resolution and digital decoding integrated processing of low-resolution intermediate coded images. Background Technology

[0002] Wavefront-coded imaging is a computational optical imaging technique that combines optical encoding and digital decoding in two steps. The wavefront-coded imaging system adds a specially shaped optical phase plate near the aperture of the optical system to encode and modulate the incident light from the scene. This results in high consistency of the intermediate encoded image output by the imaging detector over a wide range of ambient temperatures or a large depth of focus. The intermediate encoded image is then digitally decoded to obtain a clear image. Because the optical point spread function (PSF) of the wavefront-coded imaging system has a large support region on the focal plane, it helps to retain high-frequency information lost due to discrete sampling by the detector, making wavefront-coded imaging a promising candidate for super-resolution imaging.

[0003] Currently, how to perform super-resolution reconstruction and digital decoding deblurring on intermediate encoded images to obtain high-resolution, clear decoded images remains a problem to be solved. Although internationally, learning models for image super-resolution and deblurring have been proposed, existing learning models have limitations when used in wavefront-coded super-resolution imaging schemes. Specifically, these limitations include: ① Existing learning models require input to clear low-resolution images or low-resolution images containing motion-induced blur, while the input in wavefront-coded imaging schemes is optically encoded intermediate encoded images; ② Existing learning models mainly achieve high-quality super-resolution images by employing complex network architectures or increasing the number and depth of network layers, while wavefront-coded super-resolution imaging schemes require the design of lightweight neural network models. Summary of the Invention

[0004] To address the aforementioned problems in the existing technology, this invention provides an integrated super-resolution and digital decoding processing method for low-resolution intermediate coded images. The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] This invention provides a method for integrating super-resolution and digital decoding processing of low-resolution intermediate coded images, comprising:

[0006] S1: Establish an image database that includes a high-resolution clear image set, a high-resolution intermediate coded image set, and a low-resolution intermediate coded image set;

[0007] S2: Construct a super-resolution sub-network and pre-train the super-resolution sub-network using the low-resolution intermediate coded image set and the high-resolution intermediate coded image set to obtain the pre-trained super-resolution sub-network.

[0008] S3: Construct a super-resolution and digital decoding integrated network model, which includes a super-resolution sub-network and a digital decoding sub-network connected in series.

[0009] S4: Using the low-resolution intermediate coded image set and the high-resolution clear image set, train the super-resolution and digital decoding integrated network model to obtain the trained super-resolution and digital decoding integrated network model.

[0010] S5: Input the original low-resolution intermediate encoded image into the integrated network model of the trained super-resolution sub-network and digital decoding sub-network to obtain the corresponding super-resolution decoded image.

[0011] In one embodiment of the present invention, S1 includes:

[0012] S1.1: Use an infrared camera to acquire a predetermined number of clear images as a high-resolution clear image set;

[0013] S1.2: Based on the optomechanical structure parameters of the wavefront-coded optical imaging system, use optical simulation software to load the high-resolution clear image set and generate a high-resolution intermediate coded image set.

[0014] S1.3: Perform a 2×2 pixel merging operation on each image in the high-resolution clear image set to form a low-resolution intermediate coded image set.

[0015] In one embodiment of the present invention, S2 includes:

[0016] S2.1: Construct a super-resolution sub-network, which includes an image interpolation module, a residual network module, and a summing module. The image interpolation module takes a low-resolution intermediate encoded image as input and generates principal component information of the super-resolution intermediate encoded image. The residual network module takes the principal component information of the super-resolution intermediate encoded image as input and generates residual information of the super-resolution intermediate encoded image. The input of the summing module is connected to both the image interpolation module and the residual network module, and is used to sum the principal component information and the residual information to generate the super-resolution intermediate encoded image.

[0017] S2.2: Select the images in the low-resolution intermediate coded image set as sample features, and select the corresponding images in the high-resolution intermediate coded image set as labels to form training samples;

[0018] S2.3: Input the images in the set of low-resolution intermediate encoded images into the constructed super-resolution sub-network to obtain super-resolution intermediate encoded images. Based on the super-resolution intermediate encoded images and the corresponding high-resolution intermediate encoded image labels, calculate the loss value using the loss function, update the parameters of the super-resolution sub-network, and generate the pre-trained super-resolution sub-network.

[0019] S2.4: Input the images in the set of low-resolution intermediate encoded images into the pre-trained super-resolution sub-network to obtain the corresponding super-resolution intermediate encoded images.

[0020] In one embodiment of the present invention, the residual network module includes N+1 convolutional blocks connected in sequence, wherein the k-th convolutional block consists of a convolutional layer + ReLU activation function, 1≤k≤N; and the last convolutional block consists of a convolutional layer + Tanh activation function.

[0021] In one embodiment of the present invention, S2.3 includes:

[0022] The low-resolution intermediate encoded image in the training samples is input into the super-resolution sub-network for training to obtain the super-resolution intermediate encoded image.

[0023] Calculate the SSIM value of the generated super-resolution intermediate coded image and the high-resolution intermediate coded image in the training samples, and use (1-SSIM) as the loss function to calculate the loss value;

[0024] If the loss value does not converge, the parameters of the super-resolution sub-network are updated using the backpropagation algorithm based on the calculated loss value. If the loss value converges, the current super-resolution sub-network is used as the final super-resolution sub-network after pre-training.

[0025] In one embodiment of the present invention, the digital decoding subnetwork includes a Wiener filter image restoration module, a shallow convolutional neural network module, and a fusion module, wherein,

[0026] The Wiener filter image restoration module takes the super-resolution intermediate encoded image as input and uses the Wiener filter image restoration method to generate principal component information of the digital decoded image; the shallow convolutional neural network module takes the super-resolution intermediate encoded image as input and uses it to generate residual information of the digital decoded image; the fusion module performs channel splicing on the principal component information and residual information of the digital decoded image, and performs pixel-by-pixel multi-channel adaptive weighted fusion on the spliced ​​image to generate the super-resolution decoded image.

[0027] In one embodiment of the present invention, the shallow convolutional neural network module includes four interconnected convolutional blocks, wherein the first, second, and third convolutional blocks are each composed of one convolutional layer and a ReLU activation function, and the fourth convolutional block is composed of one convolutional layer and a Tanh activation function.

[0028] In one embodiment of the present invention, S4 includes:

[0029] Keeping the network parameters of the pre-trained super-resolution sub-network fixed, using the low-resolution intermediate encoded image set as samples and the high-resolution clear image set as labels, the low-resolution intermediate encoded image set is input into the constructed super-resolution and digital decoding integrated network to generate super-resolution decoded images. The SSIM index values ​​of the generated super-resolution decoded images and the corresponding high-resolution clear image labels are calculated, and the loss value is calculated using (1-SSIM) as the loss function.

[0030] If the loss value does not converge, the parameters of the digital decoding sub-network are updated using the backpropagation algorithm based on the calculated loss value. If the loss value converges, the current super-resolution and digital decoding integrated network model is used as the final trained super-resolution and digital decoding integrated network model.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] This invention establishes an image database containing three image sets: high-resolution clear images, high-resolution intermediate encoded images, and low-resolution intermediate encoded images. It trains and optimizes a super-resolution sub-network, trains and optimizes an integrated super-resolution and digital decoding network model for intermediate encoded images, and tests the optimal integrated super-resolution and digital decoding network model for intermediate encoded images. This invention provides a super-resolution and digital decoding integrated processing method for low-resolution intermediate encoded images that can effectively process blurry low-resolution intermediate encoded images into clear super-resolution decoded images, has good interpretability, and is lightweight.

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method for integrating super-resolution and digital decoding of low-resolution intermediate coded images provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of a super-resolution sub-network provided in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the structure of a digital decoding sub-network model provided in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the super-resolution and digital decoding integrated network model for low-resolution intermediate encoded images provided in an embodiment of the present invention;

[0038] Figure 5a This is a low-resolution intermediate coded image used for testing in an embodiment of the present invention;

[0039] Figure 5bThis is a super-resolution intermediate encoded image generated in an embodiment of the present invention;

[0040] Figure 5c It is a super-resolution decoded image generated in an embodiment of the present invention. Detailed Implementation

[0041] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail, with reference to the accompanying drawings and specific embodiments, a method for super-resolution and digital decoding integration processing of low-resolution intermediate coded images proposed according to the present invention.

[0042] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0044] Please see Figure 1 , Figure 1 This is a flowchart of a method for integrating super-resolution and digital decoding of low-resolution intermediate-coded images, provided by an embodiment of the present invention. The method includes:

[0045] S1: Establish an image database that includes a high-resolution clear image set, a high-resolution intermediate coded image set, and a low-resolution intermediate coded image set.

[0046] Step S1 in this embodiment specifically includes the following steps:

[0047] S1.1: Use an infrared camera to acquire a predetermined number of clear images as a high-resolution clear image set with an image resolution of 640×480;

[0048] S1.2: Based on the optomechanical structure parameters of the wavefront-coded optical imaging system, the high-resolution clear image set is loaded using optical simulation software to simulate and generate a high-resolution intermediate coded image set with an image resolution of 640×480.

[0049] S1.3: Perform a 2×2 pixel merging operation on each image of the high-resolution intermediate coded image set to form a low-resolution intermediate coded image set with an image resolution of 320×240.

[0050] In this embodiment, the image capacity of the high-resolution clear image set, the high-resolution intermediate encoded image set, and the low-resolution intermediate encoded image set in the image database is 2500 images each.

[0051] S2: Construct a super-resolution sub-network and pre-train the super-resolution sub-network using the low-resolution intermediate coded image set and the high-resolution intermediate coded image set to obtain the pre-trained super-resolution sub-network.

[0052] Specifically, step S2 in this embodiment includes:

[0053] S2.1: Construct a super-resolution sub-network, which includes an image interpolation module, a residual network module, and a summing module.

[0054] In this embodiment, the super-resolution sub-network adopts a residual network architecture, uses image interpolation methods to generate principal component information of the super-resolution intermediate coded image, and the residual network branches learn the residual information of the super-resolution intermediate coded image.

[0055] Specifically, please see Figure 2 , Figure 2 This is a schematic diagram of a super-resolution sub-network provided in an embodiment of the present invention. The super-resolution sub-network includes an image interpolation module, a residual network module, and a summing module. The image interpolation module is used to input a low-resolution intermediate coded image and generate principal component information of a super-resolution intermediate coded image. The residual network module is used to input the principal component information of the super-resolution intermediate coded image and generate residual information of the super-resolution intermediate coded image. The input of the summing module is connected to both the image interpolation module and the residual network module, and is used to sum the principal component information and the residual information to generate the super-resolution intermediate coded image.

[0056] Preferably, the image interpolation method in the image interpolation module of this embodiment is bilinear interpolation, but either nearest neighbor interpolation or bicubic interpolation can also be used.

[0057] The residual network module includes N+1 convolutional blocks connected in sequence. The kth convolutional block consists of a convolutional layer and a ReLU activation function to accelerate the convergence process, where 1≤k≤N. The last convolutional block consists of a convolutional layer and a Tanh activation function to ensure that the output data value range falls within [-1.0,+1.0].

[0058] Preferably, the residual network module of this embodiment includes three convolutional blocks connected in sequence. The first convolutional block has 1 input channel, 64 output channels, a kernel size of 9×9, and a stride of 1 pixel. The second convolutional block has 64 input channels, 32 output channels, a kernel size of 1×1, and a stride of 1 pixel. The third convolutional block has 32 input channels, 1 output channel, a kernel size of 5×5, and a stride of 1 pixel.

[0059] S2.2: Select the images in the low-resolution intermediate coded image set as sample features, and select the corresponding images in the high-resolution intermediate coded image set as labels to form training samples.

[0060] Specifically, 2000 low-resolution intermediate coded images from the low-resolution intermediate coded image set are selected as sample features, and 2000 high-resolution intermediate coded images from the high-resolution intermediate coded image set that are from the same source as the low-resolution intermediate coded images are selected as labels to form training samples.

[0061] S2.3: Input the images from the low-resolution intermediate encoded image set into the constructed super-resolution sub-network to obtain super-resolution intermediate encoded images. Based on the super-resolution intermediate encoded images and the corresponding high-resolution intermediate encoded image labels, calculate the loss value using the loss function, update the parameters of the super-resolution sub-network, and generate the pre-trained super-resolution sub-network.

[0062] Specifically, the low-resolution intermediate encoded image in the training samples is input into the super-resolution sub-network for training to obtain the super-resolution intermediate encoded image;

[0063] Calculate the SSIM (Structural Similarity) value between the generated super-resolution intermediate coded image and the high-resolution intermediate coded image in the training samples, and calculate the loss value using 1-SSIM as the loss function;

[0064] If the loss value does not converge, the parameters of the super-resolution sub-network are updated using the backpropagation algorithm based on the calculated loss value. If the loss value converges, the current super-resolution sub-network is used as the final super-resolution sub-network after pre-training.

[0065] S2.4: Input the images in the set of low-resolution intermediate encoded images into the pre-trained super-resolution sub-network to obtain the corresponding super-resolution intermediate encoded images.

[0066] S3: Construct a super-resolution and digital decoding integrated network, which includes a super-resolution sub-network and a digital decoding sub-network connected in series.

[0067] In this embodiment, the super-resolution sub-network and the digital decoding sub-network are connected in series to form an integrated super-resolution and digital decoding network. The digital decoding sub-network adopts a two-branch fusion architecture of Wiener filtering image restoration method and shallow neural network model. The first branch uses Wiener filtering image restoration method to generate principal component information of digital decoded image, and the second branch uses shallow convolutional neural network model to learn residual information of digital decoded image. The output results of the two branches are subjected to pixel-by-pixel multi-channel adaptive weighted fusion to generate super-resolution decoded image.

[0068] Specifically, please see Figure 3 , Figure 3 This is a schematic diagram of a digital decoding sub-network model provided in an embodiment of the present invention. The digital decoding sub-network includes a Wiener filter image restoration module, a shallow convolutional neural network module, and a fusion module. The Wiener filter image restoration module is used to input the super-resolution intermediate encoded image and generate principal component information of the digital decoded image using the Wiener filter image restoration method (Hasler I, Bustin N, Price D. Thermal imaging system using optimized wavefront coding operating in real-time. Proc. of SPIE, 2010, 7834:783402). The shallow convolutional neural network module is used to input the super-resolution intermediate encoded image and generate residual information of the digital decoded image. The fusion module is used to perform pixel-by-pixel multi-channel adaptive weighted fusion of the output results of the Wiener filter image restoration module and the shallow convolutional neural network module, i.e., the principal component information and residual information of the digital decoded image, to generate the super-resolution decoded image.

[0069] In this embodiment, the shallow convolutional neural network module includes four interconnected convolutional blocks. The first, second, and third convolutional blocks are each composed of one convolutional layer and a ReLU activation function to accelerate the convergence process. The last convolutional block is composed of one convolutional layer and a Tanh activation function to ensure that the output data value range falls within [-1.0, +1.0].

[0070] Preferably, in the shallow convolutional neural network module, the first convolutional block has 1 input channel, 64 output channels, a kernel size of 13×13, and a stride of 1 pixel; the second convolutional block has 64 input channels, 32 output channels, a kernel size of 3×3, and a stride of 1 pixel; the third convolutional block has 32 input channels, 32 output channels, a kernel size of 5×5, and a stride of 1 pixel; and the fourth convolutional block has 32 input channels, 63 output channels, a kernel size of 3×3, and a stride of 1 pixel.

[0071] Furthermore, the digital decoded image generated by the shallow convolutional neural network module has 63 channels, and the digital decoded image generated by the Wiener filter image restoration module has 1 channel. The fusion module concatenates the outputs of these two modules and performs pixel-by-pixel multi-channel adaptive weighted fusion on the concatenated image to generate a super-resolution decoded image.

[0072] S4: Using the low-resolution intermediate encoded image set and the high-resolution clear image set, train the super-resolution and digital decoding integrated network model to obtain the trained super-resolution and digital decoding integrated network model.

[0073] The system uses a set of low-resolution intermediate encoded images as samples and a set of high-resolution clear images as labels. Specifically, low-resolution intermediate encoded images are selected as input from the set of low-resolution intermediate encoded images, and high-resolution clear images from the set of high-resolution clear images that are from the same source as the low-resolution intermediate encoded images are selected as labels. Keeping the network parameters of the pre-trained super-resolution sub-network fixed, the low-resolution intermediate encoded image set is input into the constructed super-resolution and digital decoding integrated network. The SSIM index values ​​of the generated super-resolution decoded images and their corresponding high-resolution clear image labels are calculated, and the loss value is calculated using (1-SSIM) as the loss function.

[0074] The network parameters of the super-resolution and digital decoding integrated network are iteratively updated to generate the super-resolution and digital decoding integrated network model of the trained intermediate encoded image.

[0075] Specifically, if the loss value does not converge, the parameters of the digital decoding sub-network of the intermediate encoded image are updated using the backpropagation algorithm based on the calculated loss value; if the loss value converges, the current super-resolution and digital decoding integrated network model is used as the trained final super-resolution and digital decoding integrated network model.

[0076] S5: Input the original low-resolution intermediate encoded image into the integrated network model of the trained super-resolution sub-network and digital decoding sub-network to obtain the corresponding super-resolution decoded image.

[0077] For the trained super-resolution and digital decoding integrated network model, inputting a low-resolution intermediate encoded image will output the corresponding super-resolution decoded image.

[0078] Please see Figures 5a to 5c , Figure 5a This is a low-resolution intermediate coded image used for testing in an embodiment of the present invention; Figure 5b This is a super-resolution intermediate encoded image generated in an embodiment of the present invention; Figure 5c This refers to the super-resolution decoded image generated in this embodiment of the invention. In this embodiment, to test the effect of the super-resolution and digital decoding integrated processing method for low-resolution intermediate encoded images in this invention, the remaining 500 low-resolution intermediate encoded images in the aforementioned image database are used as sample features, and the corresponding 500 high-resolution clear images are used as labels. The test batch size is set to 5, and the images are input into the trained super-resolution and digital decoding integrated network model to obtain the processed super-resolution decoded image. The SSIM index is calculated for each super-resolution decoded image and its corresponding high-resolution clear image, and the average structural similarity index value of the 500 images is statistically analyzed, with an average value of 0.937. Figures 5a to 5c It can be seen that the super-resolution and digital decoding integrated processing method proposed in the embodiments of the present invention can effectively process blurry low-resolution intermediate encoded images into clear super-resolution decoded images.

[0079] This invention provides a super-resolution and digital decoding integrated processing method for low-resolution intermediate encoded images. By establishing an image database of three image sets—high-resolution clear images, high-resolution intermediate encoded images, and low-resolution intermediate encoded images—a super-resolution sub-network is trained and optimized, a super-resolution and digital decoding integrated network model is trained and optimized, and the optimal super-resolution and digital decoding integrated network model for intermediate encoded images is tested. This invention provides a method for super-resolution and digital decoding integrated processing of low-resolution intermediate encoded images that can effectively process blurry low-resolution intermediate encoded images into clear super-resolution decoded images, with good interpretability and lightweight design.

[0080] In the several embodiments provided by this invention, it should be understood that the apparatus and methods disclosed in this invention can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0081] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0082] Another embodiment of the present invention provides a storage medium storing a computer program for executing the steps of the integrated processing method for super-resolution and digital decoding of intermediate encoded images described in the above embodiments. A further aspect of the present invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the integrated processing method for super-resolution and digital decoding of intermediate encoded images as described in the above embodiments. Specifically, the integrated modules implemented as software functional modules can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0083] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method of integrating super-resolution and digital decoding of a low resolution intermediate coded image, characterized in that, The method comprises the following steps: S1: establishing an image database comprising a high-resolution clear image set, a high-resolution intermediate coding image set and a low-resolution intermediate coding image set; S2: constructing a super-resolution sub-network and pre-training the super-resolution sub-network using the low-resolution intermediate coding image set and the high-resolution intermediate coding image set, to obtain a pre-trained super-resolution sub-network; S3: constructing a super-resolution and digital decoding integrated network model, wherein the super-resolution and digital decoding integrated network model comprises a super-resolution sub-network and a digital decoding sub-network connected in series; S4: training the super-resolution and digital decoding integrated network model using the low-resolution intermediate coding image set and the high-resolution clear image set, to obtain a trained super-resolution and digital decoding integrated network model; S5: inputting an original low-resolution intermediate coding image into the trained super-resolution sub-network and digital decoding sub-network integrated network model, to obtain a corresponding super-resolution decoded image. The S2 comprises: S2.1: constructing a super-resolution sub-network, wherein the super-resolution sub-network comprises an image interpolation module, a residual network module and a summation module, wherein the image interpolation module inputs a low-resolution intermediate coding image, and is used to generate main component information of a super-resolution intermediate coding image, the residual network module is used to input the main component information of the super-resolution intermediate coding image, and generate residual information of the super-resolution intermediate coding image, and the input end of the summation module is connected with the image interpolation module and the residual network module respectively, and is used to sum the main component information and the residual information, and generate a super-resolution intermediate coding image; S2.2: selecting an image in the low-resolution intermediate coding image set as a sample feature, and selecting a corresponding image in the high-resolution intermediate coding image set as a label, to form a training sample; S2.3: inputting an image in the low-resolution intermediate coding image set into the constructed super-resolution sub-network to obtain a super-resolution intermediate coding image, calculating a loss value by using a loss function based on the super-resolution intermediate coding image and a corresponding high-resolution intermediate coding image label, updating parameters of the super-resolution sub-network, and generating a pre-trained super-resolution sub-network; S2.4: inputting an image in the low-resolution intermediate coding image set into the pre-trained super-resolution sub-network, to obtain a corresponding super-resolution intermediate coding image; The digital decoding sub-network comprises a Wiener filter image restoration module, a shallow convolutional neural network module and a fusion module, wherein the Wiener filter image restoration module inputs the super-resolution intermediate coding image, and generates main component information of a digital decoding image by using a Wiener filter image restoration method; the shallow convolutional neural network module inputs the super-resolution intermediate coding image, and is used to generate residual information of the digital decoding image; and the fusion module is used to channel splice the residual information and the main component information of the digital decoding image, and perform pixel-by-pixel multi-channel adaptive weighted fusion on the generated image after splicing, to generate a super-resolution decoded image.

2. The method of super-resolution and digital decoding integration processing of low-resolution intermediate coded images according to claim 1, characterized in that, The S1 comprises: S1.1: acquiring a predetermined number of clear images by using an infrared camera, as a high-resolution clear image set; S1.2: According to the optical and mechanical structure parameters of the wavefront coding optical imaging system, using optical simulation software, loading the high-resolution clear image set, generating a high-resolution intermediate coding image set; S1.3: 2x2 pixel merging operation is performed on each image of the high-resolution clear image set to form a low-resolution intermediate coding image set.

3. The method of super-resolution and digital decoding integration processing of low-resolution intermediate coded images according to claim 1, characterized in that, The residual network module comprises N+1 convolutional blocks connected in sequence, wherein the kth convolutional block is composed of one convolutional layer+ReLU activation function, 1≤k≤N; the last convolutional block is composed of one convolutional layer+Tanh activation function.

4. The method of super-resolution and digital decoding integration processing of low-resolution intermediate coded images according to claim 1, characterized in that, The S2.3 comprises: The low-resolution intermediate coding image in the training sample is input into the super-resolution subnetwork for training to obtain a super-resolution intermediate coding image; The SSIM value of the generated super-resolution intermediate coding image and the high-resolution intermediate coding image in the training sample is calculated, and (1-SSIM) is used as a loss function to calculate a loss value; In the case that the loss value does not converge, the parameters of the super-resolution subnetwork are updated based on the calculated loss value using a back propagation algorithm, and in the case that the loss value converges, the current super-resolution subnetwork is taken as a final pre-trained super-resolution subnetwork.

5. The method of super-resolution and digital decoding integration processing of low-resolution intermediate coded images according to claim 1, characterized in that, The shallow convolutional neural network module comprises four convolutional blocks connected to each other, wherein the first convolutional block, the second convolutional block and the third convolutional block are each composed of one convolutional layer+ReLU activation function, and the fourth convolutional block is composed of one convolutional layer+Tanh activation function.

6. The method of super-resolution and digital decoding integration processing of low-resolution intermediate coded images according to any of claims 1 to 5, characterized in that, The S4 comprises: The network parameters of the pre-trained super-resolution subnetwork are kept fixed, the low-resolution intermediate coding image set is taken as a sample, the high-resolution clear image set is taken as a label, the low-resolution intermediate coding image set is input into the constructed super-resolution and digital decoding integrated network to generate a super-resolution decoding image, the SSIM index value of the generated super-resolution decoding image and the corresponding high-resolution clear image label is calculated, and (1-SSIM) is used as a loss function to calculate a loss value; In the case that the loss value does not converge, the parameters of the digital decoding subnetwork are updated based on the calculated loss value using a back propagation algorithm, and in the case that the loss value converges, the current super-resolution and digital decoding integrated network model is taken as a final trained super-resolution and digital decoding integrated network model.

Images