Super-resolution reconstruction method based on extraction of low-frequency characteristics of ink-jet printing image

Abstract

The invention provides a super-resolution reconstruction method based on extraction of low-frequency characteristics of an ink-jet printing image, and the method comprises the steps: S1, building a data set, collecting images of different amplification factors, different printing dot area rates and different ink dot positions on different pieces of paper, building a data sample set, and dividing the data sample set into a training set and a test set; s2, constructing a model architecture, and using a deep residual convolutional network formed by using dense connection and jump connection modes on a generative network part of an original SRGAN network model; and S3, adjusting parameters of the model in continuous generation-confrontation of image feature information by adopting a construction mode of the model architecture and utilizing the characteristics of a generative confrontation network of the SRGAN network model to obtain a reconstruction effect of a reconstructed image. According to the method, different parts of the input data are dynamically paid attention to by adding the convolution block attention module CBAM, the understanding and generation capability of the model is improved, and the resolution of the image is improved.

Description

Technical Field

[0001] The present invention relates to the fields of image processing, deep learning technology, and computer vision, and particularly relates to a super-resolution reconstruction method based on extracting low-frequency features of inkjet-printed images. Background Art

[0002] In the process of inkjet printing, due to the lack of sufficient scene and appearance information in some low-resolution images, some information in the results is blurred and the results are not satisfactory. In addition, there are some problems when using high-resolution inkjet-printed images for printing, especially the low printing recognition accuracy of high-resolution inkjet-printed images, the inability to accurately extract edge information, and the inability to accurately distinguish different printed patterns, which also seriously restricts the popularization and development of inkjet printing. And images are the data mode that most conforms to human intuitive vision. With the development of technology, inkjet printing uses image data as the form of data input and data output. How to improve the accurate input of the quality of the input printed image to print an accurate printed image is the key problem of inkjet printing. In response to this problem, the present invention proposes a super-resolution reconstruction method based on extracting low-frequency features of inkjet-printed images. Thus, the accurate input of the quality of the input printed image is improved to print an accurate printed image. Therefore, improving image quality has become an important goal in the field of computer image processing. At present, the powerful non-linear ability and feature learning ability of deep learning have become the mainstream methods in super-resolution reconstruction.

[0003] For example, Chinese Patent No. CN111932456A discloses a single-image super-resolution reconstruction method based on a generative adversarial network. The invention designs a generative adversarial network using a simplified residual connection, which consists of a generator network and a discriminator network for reconstructing low-resolution images. The generator network does not use a normalization layer, but it still uses a simple residual connection of the addition method, and the effect of feature extraction for low-resolution images is still not detailed enough. The generator cannot learn more advanced image features, resulting in poor effects of the generated high-resolution images.

[0004] For example, a Chinese patent with the publication number CN116721018A discloses an image super-resolution reconstruction method based on a generative adversarial network with dense residual connections. The invention designs a self-attention generative adversarial network composed of dense residual blocks, which consists of multiple layers of generator networks and multiple layers of discriminator networks. Although it uses dense residual block connections with an additive method, the increase in convolutional depth will increase the computational amount and lengthen the training time. Moreover, the discriminator does not extract the high-frequency information features in the reconstructed image or the high-resolution image in detail, making it impossible for the generator to learn more advanced image features and obtain better high-resolution image effects. Therefore, a super-resolution reconstruction method for extracting low-frequency features of inkjet printed images is needed to solve the problems such as the insufficient extraction and utilization of low-frequency feature information of pictures in the prior art, and thus achieve better high-resolution image effects. Summary of the Invention

[0005] The present invention aims to provide a super-resolution reconstruction method based on extracting low-frequency features of inkjet printed images to solve the problems such as the insufficient extraction and utilization of low-frequency feature information of pictures in the prior art. The technical problems to be solved by the present invention are achieved through the following technical solutions.

[0006] The present invention provides a super-resolution reconstruction method based on extracting low-frequency features of inkjet printed images. The main purpose is to extract the low-frequency feature information of the input image to obtain more fine features of the inkjet printed image. The method includes the following steps:

[0007] Step S1: Establishment of the data set: Collect images with different magnification ratios, different printing dot area ratios, and different ink dot positions on different papers, establish a data sample set, and divide it into a training set and a test set.

[0008] Step S2: Model architecture construction: In the generator network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connection and skip connection methods to fully extract the low-frequency features and high-frequency features of the input image, and then generate a new reconstructed image to make it have the same background features as the input image; in the discriminator network part of the original SRGAN network model, use DWT (Discrete Wavelet Transform) to extract the high-frequency information of the reconstructed super-resolution image or the collected high-resolution image, and fuse it with the corresponding medium-low frequency information, and then perform normalization processing to obtain more minute features in the details of the image. And the convolutional block attention module CBAM dynamically focuses on different parts of the input data to improve the model's understanding and generation ability, so as to improve the resolution of the image.

[0009] Step S3: Adopt the above model architecture construction method, and use the characteristics of the generative adversarial network (GAN) of the SRGAN network model to adjust the parameters of the model during the continuous generation and confrontation of image feature information, and obtain better reconstruction effects of the reconstructed image.

[0010] Step S1: Establishment of the data set, specifically including the following steps:

[0011] Use a CCD camera to collect high-resolution images of the paper surface after inkjet printing. Ensure that the data set covers various printing conditions such as different magnification ratios (455, 1024 times), different printing dot area ratios (2%-100%), and different ink dot positions on different papers to ensure the diversity of the data.

[0012] Step S2: Construction of the model architecture, specifically including the following steps:

[0013] Step S2-1: In the generation network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connection and skip connection methods. By using the method of residual convolution, obtain more original input feature information of the input image. Except for the first and the last convolutional modules (Conv), the input of each convolutional module is the output of all previous modules, and its output is the input of all subsequent convolutional modules. The output of the i-th convolutional module is:

[0014] L i+1 = S×(L i + L i-1 +…+ L2 + x) (1)

[0015] where L i+1 is the output of the i-th convolution except for the first and the last convolutional modules (Conv). The input of the last convolutional module is the sum of the output of the first convolutional module and the output of the sixth convolutional module. Starting from the second convolutional module, L i + L i-1 +…+ L2 represents the sum of the outputs of the (i - 1)-th convolutional module to the second convolutional module. S is the relevant mechanism function of the convolution process, and x is the input feature map. Since the number of channels in the output layer of the dense connection network will gradually accumulate, the last convolution uses a 1×1 convolutional layer for dimensionality reduction. Therefore, the output of the entire deep residual convolutional network is:

[0016] F outpit = W L f contact (L l+1 + L l + L l-1 +…+ L2 + x) (2)

[0017] where F output is the output of the entire deep residual convolutional network, x is the input of the dense residual feature aggregation module, W L is the weight of the last convolutional layer, and f contact (Ll+1 +L l +L l-1 +…+L2 + x) is a feature fusion operation. Through the process of feature fusion and dimensionality reduction, not only can the computational complexity during model training be reduced, but it also helps to extract more feature information for the subsequent network.

[0018] Step S2 - 2: In the discriminator network part of the original SRGAN network model, use DWT (Discrete Wavelet Transform) to extract the high - frequency information of the reconstructed super - resolution image or the collected high - resolution image, fuse it with the corresponding mid - low - frequency information, and then perform normalization processing to obtain more low - frequency and high - frequency feature information of the image. Its formula is expressed as:

[0019] I DWT = deal(αLH + βHL + γHH) (3)

[0020] Among them, HL, LH, and HH are three sub - bands representing high - frequency information, α, β, and γ are balance factors, which are set to 0.7, 0.8, and 1.2 respectively, and deal(*) is a data processing operation.

[0021] The beneficial effects of the present invention are as follows:

[0022] The present invention provides a super - resolution reconstruction method based on extracting low - frequency features of ink - jet printed images to solve problems such as the insufficient extraction and utilization of low - frequency feature information of ink - jet printed images in the prior art. In the generator network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connection and skip connection methods to fully extract the low - frequency and high - frequency features of the input image, and then generate a new reconstructed image, making it have the same background features as the input image; in the discriminator network part of the original SRGAN network model, use DWT (Discrete Wavelet Transform) to extract the high - frequency information of the reconstructed super - resolution image or the collected high - resolution image, fuse it with the corresponding mid - low - frequency information, and then perform normalization processing to obtain more minute features in the details of the image. At the same time, in order to enable the model to effectively capture important information, by adding a convolutional block attention module CBAM to dynamically focus on different parts of the input data, the understanding and generation ability of the model are improved to enhance the resolution of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic structural diagram of the deep residual convolutional network of the present invention;

[0024] Figure 2 is a schematic structural diagram of the channel attention module and the spatial attention module of the present invention;

[0025] Figure 3It is a schematic structural diagram of the discriminator network including a channel attention module and a spatial attention module of the present invention;

[0026] Figure 4 It is a comparison diagram of the method effects of the present invention. Detailed implementation manners

[0027] The following will combine with the attached Figures 1-4 The technical solutions of the present invention will be described in detail. However, the embodiments described through the drawings are exemplary and are only used to explain the present invention and cannot limit the scope of the rights of the present invention.

[0028] This embodiment provides a super-resolution reconstruction method based on extracting low-frequency features of inkjet printed images, specifically including the following steps:

[0029] Step S1: Establish a data set, collect images with different magnification ratios, different printing dot area ratios, and different ink dot positions on different papers, establish a data sample set, and divide it into a training set and a test set.

[0030] In step S1, a CCD camera is used to collect high-resolution images on the surface of the paper after inkjet printing; ensure that the data set covers various printing conditions with different magnification ratios from 455 times to 1024 times, different printing dot area ratios from 2% to 100%, and different ink dot positions on different papers to ensure the diversity of the data.

[0031] Step S2: Construct a model architecture. In the generation network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connections and skip connections to fully extract the low-frequency features and high-frequency features of the input image, and then generate a new reconstructed image to make it have the same background features as the input image;

[0032] As Figure 1 shown is a schematic structural diagram of the deep residual convolutional network of the present invention. By using the methods of dense connection and skip connection, the convolutional modules are connected in an orderly manner to learn more feature information and effectively fuse the low-level and high-level features of different layers, while alleviating the problems of gradient disappearance and gradient explosion and completing the effect accumulation of the high-frequency features extracted by the designed module.

[0033] The specific steps are as follows:

[0034] Step S21: In the generation network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connections and skip connections, as Figure 1As shown. By using the residual convolution method, more original input feature information of the input image is obtained; except for the first and the last convolution modules, the input of each convolution module is the output of all previous modules, and its output is the input of all subsequent convolution modules; the output of the i-th convolution module is:

[0035] L i+1 = S×(L i + L i-1 +…+ L2 + x) (1)

[0036] where, L i+1 is the output of the i-th convolution except for the first and the last convolution modules, and the input of the last convolution module is the sum of the output of the first convolution module and the output of the sixth convolution module;

[0037] Starting from the second convolution module, L i + L i-1 +…+ L2 represents the sum of the outputs of the (i - 1)-th convolution module to the second convolution module; S is the relevant mechanism function in the convolution process, and x is the input feature map;

[0038] Since the number of channels in the output layer of the dense connection network will gradually accumulate, the last convolution uses a 1×1 convolution layer for dimensionality reduction; therefore, the output of the entire deep residual convolution network is:

[0039] F output = W L f contact (L l+1 + L l + L l-1 +…+ L2 + x) (2)

[0040] where, F output is the output of the entire deep residual convolution network, x is the input of the dense residual feature aggregation module, W L is the weight of the last convolution layer, and f contact (L l+1 + L l + L l-1 +…+ L2 + x) is the feature fusion operation; by the process of feature fusion and dimensionality reduction, the computational amount in the model training process is reduced, and more feature information is extracted for the subsequent network;

[0041] Such as Figure 2The following is a schematic diagram of the channel attention module and the spatial attention module of the present invention. By utilizing the characteristics of the attention mechanism module CBAM, which first performs channel attention and then spatial attention, the information of specific target regions of interest in the image is enhanced while the information of irrelevant background regions is weakened. The channel attention module extracts the correlation between different channels, and uses the method of deep network learning to automatically obtain the importance of each feature channel, thereby strengthening the important features in the image and suppressing the unimportant features. Then, the spatial attention is used to enhance the feature expression of the key regions, so as to pay attention to the influence of global information on target pixels while focusing on local information.

[0042] As Figure 2 shown, in the channel attention module, the scaling module is the scaling ratio, which is used to control the number of neural network neurons in the middle of two fully connected layers, generally set to 16, and can be fine-tuned according to needs. Then, 1x1 convolution is performed to extract channel feature information. And the output results of the average pooling and max pooling branches are added together, and the added result is activated by Sigmoid, so as to obtain a channel attention map that can more effectively retain more useful feature information.

[0043] In the spatial attention module, the spatial attention module performs global average pooling and global max pooling operations based on channels, generating two feature maps representing different information. The concatenation in the channel attention module is to concatenate the average value and the maximum value tensors in the channel dimension, and the output size is 2×H×W. After merging, feature fusion is performed through a 7×7 convolution with a larger receptive field, and finally, a weight map is generated through a Sigmoid operation and superimposed back on the original input feature map, so that the target region is enhanced. C is the channel of the feature map, and W and H respectively represent the dimensions of the input feature map.

[0044] As Figure 3 shown, in the discriminator network part of the original SRGAN network model, the discrete wavelet transform is used to extract the high-frequency information of the reconstructed super-resolution image or the collected high-resolution image, and fuse it with the corresponding middle-low frequency information, and then perform normalization processing to obtain the minute features of the image; then, the attention module force mechanism module CBAM is used to adjust the module parameters, dynamically focus on different parts of the input data, and improve the understanding and generation ability of the model to improve the resolution of the image.

[0045] The specific steps are as follows:

[0046] Step S22: In the discriminator network part of the original SRGAN network model, the discrete wavelet transform is used to extract the high-frequency information of the reconstructed super-resolution image or the collected high-resolution image, and fuse it with the corresponding middle-low frequency information, and then perform normalization processing to obtain more low-frequency and high-frequency feature information of the image; its formula is expressed as:

[0047] I DWT = deal(αLH + βHL + γHH) (3)

[0048] Among them, HL, LH, and HH are three sub-bands representing high-frequency information, α, β, and γ are balance factors, which are set to 0.7, 0.8, and 1.2 respectively, and deal(*) is a data processing operation.

[0049] Step S3: Adopt the above model architecture construction method, utilize the characteristics of the generative adversarial network of the SRGAN network model, and adjust the parameters of the model during the continuous generation and confrontation of image feature information to obtain the reconstruction effect of the reconstructed image.

[0050] As Figure 4 shown, this embodiment respectively uses the SRGAN method and the super-resolution reconstruction method based on extracting the low-frequency features of inkjet printed images for super-resolution reconstruction, calculates the evaluation indicators PSNR and SSIM corresponding to each method respectively, and the comparison results are shown in Table 1:

[0051] SRGAN Adding a deep residual convolutional network The method provided by this solution PSNR 32.962 33.022 33.315 SSIM 0.888 0.892 0.895

[0052] As Figure 4 shown, Figure 4 in (a) is the input low-resolution inkjet printed image, Figure 4 in (b) is the effect diagram of super-resolution reconstruction using the SRGAN method, Figure 4 in (c) is the effect diagram of the super-resolution reconstruction method based on extracting the low-frequency features of inkjet printed images provided by the present invention.

[0053] In summary, a super-resolution reconstruction method based on extracting the low-frequency features of inkjet printed images is better for enhancing the clarity of inkjet printed images. By constructing a deep residual convolutional network, dense connections and skip connections are used to strengthen the acquisition of feature information by the image. At the same time, in the discriminant network part of the original SRGAN network model, DWT (Discrete Wavelet Transform) is used to extract the high-frequency information of the reconstructed super-resolution image or the acquired high-resolution image, and fuse it with the corresponding middle and low-frequency information, and then perform normalization processing to obtain more minute features in the details of the image. At the same time, in order to enable the model to effectively capture important information, by adding a convolutional block attention module CBAM to dynamically focus on different parts of the input data, while accelerating the training speed of the model and reducing the computational amount, the understanding and generation ability of the model are improved to enhance the resolution of the image.

Claims

1. A super-resolution reconstruction method based on extracting low-frequency features of inkjet printed images, characterized in that: Step S1: Establish a data set, collect images with different magnification ratios, different printing dot area ratios, and different ink dot positions on different papers, establish a data sample set, and divide it into a training set and a test set; Step S2: Construct a model architecture. In the generator network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connection and skip connection methods to fully extract the low-frequency features and high-frequency features of the input image, and then generate a new reconstructed image to make it have the same background features as the input image; In the discriminator network part of the original SRGAN network model, use discrete wavelet transform to extract the high-frequency information of the reconstructed super-resolution image or the collected high-resolution image, and fuse it with the corresponding mid-low frequency information, and then perform normalization processing to obtain the minute features of the image; Step S3: Adopt the construction method of the above model architecture, utilize the characteristics of the generative adversarial network of the SRGAN network model, adjust the parameters of the model in the continuous generation-adversary of image feature information, and obtain the reconstruction effect of the reconstructed image.

2. The super-resolution reconstruction method based on extracting low-frequency features of inkjet printed images according to claim 1, characterized in that: In step S1, use a CCD camera to collect high-resolution images on the surface of the paper after inkjet printing; ensure that the data set covers various printing conditions with different magnification ratios from 455 times to 1024 times, different printing dot area ratios from 2% to 100%, and different ink dot positions on different papers to ensure the diversity of data.

3. The super-resolution reconstruction method based on extracting low-frequency features of an inkjet printed image according to claim 1, characterized in that: The specific steps of step S2 include the following steps: Step S21: In the generator network part of the original SRGAN network model, use a deep residual convolutional network composed of dense connection and skip connection methods, and use the method of residual convolution to obtain more original input feature information of the input image; except for the first and last convolutional modules, the input of each convolutional module is the output of all previous modules, and its output is the input of all subsequent convolutional modules; the output of the i-th convolutional module is: L i+1 = S × (L i + L i-1 + … + L2 + x) (1) where L i+1 is the output of the i-th convolution except for the first and the last convolutional modules, and the input of the last convolutional module is the sum of the output of the first convolutional module and the output of the sixth convolutional module; Starting from the second convolutional module, L i + L i-1 + … + L2 represents the sum of the outputs from the (i - 1)-th convolutional module to the second convolutional module; S is the relevant mechanism function for the convolutional process, and x is the input feature map; Since the number of channels in the output layer of the dense connection network will gradually accumulate, the last convolution uses a 1×1 convolutional layer for dimensionality reduction; therefore, the output of the entire deep residual convolutional network is: F output = W L f contact (L l+1 + L l + L l-1 + … + L2 + x) (2) Among them, F output is the output of the entire deep residual convolutional network, x is the input of the dense residual feature aggregation module, and W L is the weight of the last convolutional layer, and f contact (L l+1 +L l +L l-1 +…+L2+x) is a feature fusion operation; the computational amount during the model training process is reduced through the processes of feature fusion and dimensionality reduction, and more feature information is extracted for the subsequent network; Step S22: In the discriminator network part of the original SRGAN network model, use discrete wavelet transform to extract the high-frequency information of the reconstructed super-resolution image or the collected high-resolution image, and fuse it with the corresponding mid-low frequency information, and then perform normalization processing to obtain more low-frequency and high-frequency feature information of the image; its formula is expressed as: I DWT = deal(αLH + βHL + γHH) (3) Where HL, LH, and HH are three sub-bands representing high-frequency information, α, β, and γ are balance factors, which are set to 0.7, 0.8, and 1.2 respectively, and deal(*) is a data processing operation.

Images