A method for realizing super-resolution in automation equipment

Abstract

The present invention discloses a method for realizing super-resolution in automation equipment, aiming to reconstruct and obtain high-quality and high-resolution images without increasing hardware costs, thereby improving the positioning accuracy of the visual module, improving the sorting success rate of the ultra-small component sorting equipment, or improving the positioning accuracy of the visual module, and improving the detection success rate of small-size defects of the AOI equipment. The present invention comprises the following steps: step S1: creating a sample data set S; step S2: constructing a generator model; step S3: constructing a discriminator model; step S4: creating a VGG model; step S5: training the model; step S6: deploying the trained model between the camera module and the visual positioning module for accurate sorting of ultra-small components or for visual detection of small-size defects. The present invention is applied to the technical field of automation equipment that needs to realize super-resolution.

Description

Technical Field

[0001] The present invention relates to a method for realizing super-resolution, and in particular to a method for realizing super-resolution applied in automation equipment. Background Art

[0002] Background 1: Visual sorting equipment can sort components of different categories or qualities into different containers. The sorting equipment uses a camera module to capture images of the equipment workspace. The visual module locates the position of the component in the image and converts it into the position of the component in the workspace, guiding the sorting device to complete the component sorting. Its workflow is as follows: Figure 1 As shown in the figure, for ultra-small components, the positioning accuracy of the visual module is an important bottleneck of the system. This is because the number of pixels occupied by ultra-small components in the image is too small, and the features of the components cannot be fully presented in the image, making it difficult for the visual module to locate them.

[0003] Background 2: With the development of automation technology, automated optical inspection (AOI) technology has gradually replaced manual defect detection. Compared with manual defect detection, AOI detection technology is not affected by subjective factors and can use quantifiable unified standards for detection, effectively ensuring the quality of the product. AOI detection technology requires the use of industrial cameras and industrial light sources to collect images and apply digital image processing algorithms to complete defect detection. Digital image processing methods include traditional image processing methods and image processing methods based on deep learning. These methods detect the unique features of defects in the image to complete the identification and location of defects. The workflow is as follows: Figure 2 As shown in the figure, product surface defects usually have various shapes and large size differences. Since the field of view of the image acquisition system needs to be compatible with products of various sizes, for defects of smaller sizes, there are insufficient identifiable features in the image, which can easily lead to defects that cannot be accurately detected by AOI equipment.

[0004] In order to solve the problems faced by the above two backgrounds, it is necessary to provide the visual module with higher resolution input images. There are currently two main ways to obtain high-resolution images. One is to improve the accuracy of the camera module and directly collect high-resolution images; the other is to reconstruct low-resolution images (Low-Resolution, LR) into high-resolution images (High-Resolution, HR) through super-resolution methods (Super-Resolution, SR). Using high-precision camera modules will significantly increase the production cost of the equipment and reduce the market competitiveness of the equipment. Therefore, reconstructing high-resolution images through super-resolution methods has become a more economical and practical design solution.

[0005] Super-resolution methods can be divided into interpolation-based methods and learning-based methods. Interpolation-based methods interpolate the grayscale or color of image pixels according to prior knowledge. The available methods include bilinear interpolation, bicubic interpolation, and interpolation methods with edge-preserving characteristics. This type of method is usually fast, but the texture of the reconstructed high-resolution image is relatively blurred. Early learning-based methods were mostly dictionary-based methods. Dictionary-based methods cut images into small images and group them, establish a dictionary based on the correspondence between low-resolution small images and high-resolution small images, and use the dictionary to reconstruct the newly collected low-resolution image. If the collected low-resolution image is not similar to the image in the dictionary, the reconstructed high-resolution image will be distorted. With the development of deep learning, super-resolution methods based on deep learning have occupied an important position in learning-based methods. This type of method learns the relationship between low-resolution images and high-resolution images through model training, and obtains a model that can be used to reconstruct high-resolution images. Super-resolution methods based on deep learning directly optimize the generation results of the model during the training process, without the need for artificial prior or design features, and thus have better robustness.

[0006] Although many super-resolution methods based on deep learning have been proposed, there are still the following problems when using these methods: 1. Some methods only focus on the similarity between the reconstruction result of the current image and its corresponding high-resolution image during the model training process. This can easily lead to overfitting of the model, poor generalization performance, and inability to be applied to newly acquired images; 2. Another part of the methods only focuses on the similarity between the reconstruction result of the current image and the image in the dataset during the model training process. This results in the reconstructed high-resolution image not being able to accurately restore the details in the image. Summary of the invention

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for realizing super-resolution applied in automation equipment, aiming to reconstruct and obtain high-quality and high-resolution images without increasing hardware costs, thereby improving the positioning accuracy of the visual module, improving the sorting success rate of ultra-small component sorting equipment, or improving the positioning accuracy of the visual module, and improving the detection success rate of small-size defects of AOI equipment; in addition, the present invention can achieve plug-and-play, that is, only the model needs to be deployed between the camera module and the visual positioning module to improve the positioning accuracy of the visual inspection module or the AOI inspection accuracy of the visual inspection module.

[0008] The technical solution adopted by the present invention is: the method for realizing super-resolution in an automated device comprises the following steps:

[0009] Step S1: Create a sample dataset S, collect low-resolution images of the equipment working area and the corresponding high-resolution images in pairs, S = {(LR i , HR i )}, where LR i and HR i denote the low-resolution image and the corresponding high-resolution image of the i-th ultra-small component in the data set S respectively;

[0010] Step S2: construct a generator model. The generator G consists of multiple convolution modules, nonlinear modules and upsampling modules, which is responsible for reconstructing the input low-resolution image into the corresponding high-resolution image;

[0011] Step S3: Construct a discriminator model. The discriminator D consists of multiple convolution modules, nonlinear modules, and downsampling modules. It is responsible for determining whether the input image is a reconstructed image by the generator or a real high-resolution image during the training process.

[0012] Step S4: Create a VGG model, which extracts image features for calculating model training loss;

[0013] Step S5: training the model, setting the optimizers of the generator G and the discriminator D to Adam and SGD respectively, setting the learning rate scheduling scheme to cosine simulated annealing, and training the generator G and the discriminator D alternately;

[0014] Step S6: Deploy the trained model between the camera module and the visual positioning module for accurate sorting of ultra-small components or for visual inspection of small-sized defects.

[0015] Furthermore, the equipment in step S1 is an ultra-small component sorting equipment, or an AOI optical inspection equipment for small-size defect detection.

[0016] Furthermore, the generator G constructed in step S2 includes a first convolutional layer, a first residual block, a second residual block, a second convolutional layer, a first BN layer, a third residual block, a first upsampling block, a second upsampling block and a third convolutional layer; after the low-resolution image is input from the first convolutional layer, it passes through the first residual block, the second residual block, the second convolutional layer, the first BN layer, the first residual layer, the first upsampling block, the second upsampling block and the third convolutional layer in sequence, and finally outputs a high-resolution image from the third convolutional layer; the signal output end of the first convolutional layer is divided into two paths, one path enters the first residual block, and the other path enters the first residual layer.

[0017] Furthermore, the modules of the generator G are:

[0018] The first convolutional layer: the number of input channels is 3, the number of output channels is 64, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used. This layer converts the 3-channel color image channel of the input image into 64 channels;

[0019] First residual block: the number of input channels is 64;

[0020] Second residual block: the number of input channels is 64;

[0021] Second convolutional layer: The number of input channels and output channels are both 64, the convolution kernel size is 3×3, the padding is set to 1, and the PReLU activation function is used;

[0022] The first BN layer: normalizes the convolutional data and sets the number of features to 64;

[0023] The first residual layer: performs pixel-level addition, with 64 input channels;

[0024] First upsampling layer: the number of input channels is 64;

[0025] Second upsampling layer: the number of input channels is 64;

[0026] The third convolutional layer: the number of input channels is 64, the number of output channels is 3, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used; this layer converts the feature map into 3 channels to obtain the final high-resolution image.

[0027] Furthermore, the first residual block and the second residual block each include a convolutional layer 1, a BN layer 1, a convolutional layer 2, a BN layer 2 and a residual layer 1 in sequence, wherein:

[0028] Convolutional layer 1: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PreLU activation function is used;

[0029] BN layer 1: normalize the convolutional data, and the number of features is channels;

[0030] Convolutional layer 2: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PReLU activation function is used;

[0031] BN layer 2: normalize the convolutional data, and the number of features is channels;

[0032] Residual layer 1: Residual learning is implemented by adding the input to the convolutional and batch normalized output.

[0033] Furthermore, the first upsampling layer and the second upsampling layer both include a convolutional layer A and a Pixel Shuffler layer, wherein:

[0034] Convolutional layer A: The convolution kernel size is 3×3, padding is set to 1, the number of input channels is in_channels, the number of output channels is in_channels×up_scale2, and the PreLU activation function is used to expand each channel in the input feature map to in_channels×up_scale2 channels. This operation will prepare data for the subsequent Pixel Shuffler layer;

[0035] Pixel Shuffler layer: performs an upsampling operation, rearranging the pixels in the feature map to increase the resolution.

[0036] Furthermore, the discriminator D in step S3 includes convolution layer I, convolution layer II, BN layer I, convolution layer III, BN layer II, and fully connected layer in sequence, wherein:

[0037] Convolutional layer I: The number of input channels is 3, the number of output channels is 64, the convolution kernel size is 3×3, the padding is set to 1, the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0038] Convolutional layer II: The number of input channels is 64, the number of output channels is 64, the convolution kernel size is 3×3, the stride is set to 2, and the padding is set to 1; the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0039] BN layer I: the number of features is set to 64;

[0040] Convolutional layer III: the number of input channels is 64, the number of output channels is 128, the convolution kernel size is 3×3, the stride is set to 1, and the padding is set to 1; BN layer: the number of features is set to 128; the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0041] BN layer II: the number of features is set to 64;

[0042] Fully connected layer: Use Conv2d, the number of input channels is 512, the number of output channels is set to 1024, the convolution kernel is set to 1×1, the sigmoid activation function is used, and the judgment result is output.

[0043] Furthermore, in step S5, each training cycle first sets the learning rate of the optimizer, and then performs alternating training of the generator and the discriminator. The specific content of one training is as follows:

[0044] ①Train the discriminator: First, take the real high-resolution image HR i Pass it into the discriminator D to get D_real_loss, and then pass the real low-resolution image LR i After passing the image into the generator, it is passed into the discriminator to get D_fake_loss, and finally D_train_loss is obtained as:

[0045] D_train_loss = D_real_loss + D_fake_loss;

[0046] ② Train the generator: pass the real image into the generator to get the result, and obtain the image error image_loss through the mean square error between the generator output image and the real image; then pass the generated image into the discriminator to calculate the adversarial loss, that is, the probability that the discriminator will identify the generated image as a real image; finally, pass the generated image and the real image into the VGG19 model and calculate the mean square error to obtain perception_loss. The final calculation of the generator's G_train_loss is:

[0047] G_train_loss = image_loss + 10 -3 ×adversarial loss + 2×10 -6 ×perception_loss;

[0048] Among them, image_loss and perception_loss reflect the pixel and texture differences between the reconstructed image and the real high-resolution image respectively, and adversarial loss reflects the distribution pixel degree between the reconstructed image and the real high-resolution image in the entire data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the workflow diagram of the visual sorting equipment;

[0050] Figure 2 This is the AOI equipment workflow diagram;

[0051] Figure 3 It is a schematic diagram of the model structure of the generator G;

[0052] Figure 4 It is a schematic diagram of the residual block structure;

[0053] Figure 5 It is a schematic diagram of the upsampling block structure;

[0054] Figure 6 It is a schematic diagram of the structure of the discriminator D;

[0055] Figure 7 This is a working principle diagram of the super-resolution model used for ultra-small component sorting;

[0056] Figure 8 This is a working principle diagram of the super-resolution model used for defect feature enhancement. DETAILED DESCRIPTION Embodiment 1

[0057] Through this example, it is possible to reconstruct high-quality and high-resolution images without increasing hardware costs, improve the positioning accuracy of the visual module, and increase the sorting success rate of ultra-small component sorting equipment. In addition, by deploying the model between the camera module and the visual positioning module, the positioning accuracy of the visual inspection module can be improved.

[0058] In this embodiment, the present invention is a method for realizing super-resolution in an ultra-small component sorting device, comprising the following steps:

[0059] Step S1: Create a sample dataset S. Collect low-resolution images and corresponding high-resolution images of ultra-small components in the equipment working area in pairs, S = {(LRi, HRi)}, where LRi and HRi represent the low-resolution image and corresponding high-resolution image of the i-th ultra-small component in the dataset S, respectively.

[0060] Step S2: Construct a generator model G. The generator G consists of multiple convolution modules, nonlinear modules, and upsampling modules, and is responsible for reconstructing the input low-resolution image into the corresponding high-resolution image.

[0061] Step S3: Construct a discriminator model D. The discriminator D consists of multiple convolution modules, nonlinear modules, and downsampling modules, and is responsible for determining whether the input image is an image reconstructed by the generator or a real high-resolution image during the training process.

[0062] Step S4: Create a VGG model. The VGG model extracts image features for calculating model training loss.

[0063] Step S5: Training the model. Set the optimizers of the generator and discriminator to Adam and SGD respectively, set the learning rate scheduling scheme to cosine simulated annealing, and train the generator G and the discriminator D alternately.

[0064] Step S6: Deploy the trained model between the camera module and the visual positioning module for accurate sorting of ultra-small components.

[0065] In step 2, the specific implementation of generator G is as follows Figure 3 As shown in the figure, the broken line arrows point to the residual layers of each module. The specific definitions of each module are as follows:

[0066] The first convolutional layer: the number of input channels is 3, the number of output channels is 64, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used. This layer converts the 3-channel color image channel of the input image into 64 channels;

[0067] First residual block: the number of input channels is 64;

[0068] Second residual block: the number of input channels is 64;

[0069] Second convolutional layer: The number of input channels and output channels are both 64, the convolution kernel size is 3×3, the padding is set to 1, and the PReLU activation function is used;

[0070] The first BN layer: normalizes the convolutional data and sets the number of features to 64;

[0071] The first residual layer: performs pixel-level addition, with 64 input channels;

[0072] First upsampling layer: the number of input channels is 64;

[0073] Second upsampling layer: the number of input channels is 64;

[0074] The third convolutional layer: the number of input channels is 64, the number of output channels is 3, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used; this layer converts the feature map into 3 channels to obtain the final high-resolution image.

[0075] Among them, the structure of the residual block is as follows Figure 4 As shown, the specific implementation of each layer is:

[0076] Convolutional layer 1: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PreLU activation function is used;

[0077] BN layer 1: normalize the convolutional data, and the number of features is channels;

[0078] Convolutional layer 2: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PReLU activation function is used;

[0079] BN layer 2: normalize the convolutional data, and the number of features is channels;

[0080] Residual layer 1: Residual learning is implemented by adding the input to the convolutional and batch normalized output.

[0081] The specific structure of the upsampling block is as follows Figure 5 As shown, the specific implementation of each layer is:

[0082] Convolutional layer A: The convolution kernel size is 3×3, padding is set to 1, the number of input channels is in_channels, the number of output channels is in_channels×up_scale2, and the PreLU activation function is used to expand each channel in the input feature map to in_channels×up_scale2 channels. This operation will prepare data for the subsequent Pixel Shuffler layer;

[0083] Pixel Shuffler layer: performs an upsampling operation, rearranging the pixels in the feature map to increase the resolution.

[0084] In step 3, the specific implementation of the discriminator D is as follows Figure 6 As shown, the specific definitions of each module are as follows:

[0085] Convolutional layer I: The number of input channels is 3, the number of output channels is 64, the convolution kernel size is 3×3, the padding is set to 1, the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0086] Convolutional layer II: The number of input channels is 64, the number of output channels is 64, the convolution kernel size is 3×3, the stride is set to 2, and the padding is set to 1; the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0087] BN layer I: the number of features is set to 64;

[0088] Convolutional layer III: the number of input channels is 64, the number of output channels is 128, the convolution kernel size is 3×3, the stride is set to 1, and the padding is set to 1; BN layer: the number of features is set to 128; the Leaky ReLU activation function is used, and the slope is set to 0.2;

[0089] BN layer II: the number of features is set to 64;

[0090] Fully connected layer: Use Conv2d, the number of input channels is 512, the number of output channels is set to 1024, the convolution kernel is set to 1×1, the sigmoid activation function is used, and the judgment result is output.

[0091] In this embodiment, in step S5, each training cycle first sets the learning rate of the optimizer, and then performs alternating training of the generator and the discriminator. The specific content of one training is as follows:

[0092] ①Train the discriminator: First, take the real high-resolution image HRi Pass it into the discriminator D to get D_real_loss, and then pass the real low-resolution image LR i After passing the image into the generator, it is passed into the discriminator to get D_fake_loss, and finally D_train_loss is obtained as:

[0093] D_train_loss = D_real_loss + D_fake_loss;

[0094] ② Train the generator: pass the real image into the generator to get the result, and obtain the image error image_loss through the mean square error between the generator output image and the real image; then pass the generated image into the discriminator to calculate the adversarial loss, that is, the probability that the discriminator will identify the generated image as a real image; finally, pass the generated image and the real image into the VGG19 model and calculate the mean square error to obtain perception_loss. The final calculation of the generator's G_train_loss is:

[0095] G_train_loss = image_loss + 10 -3 ×adversarial loss + 2×10 -6 ×perception_loss;

[0096] Among them, image_loss and perception_loss reflect the pixel and texture differences between the reconstructed image and the real high-resolution image respectively, and adversarial loss reflects the distribution pixel degree between the reconstructed image and the real high-resolution image in the entire data set.

[0097] In this embodiment, the present invention can achieve super-resolution for plug-and-play and fine sorting of ultra-small components, and its principle diagram is as follows: Figure 7 As shown in the figure. The super-resolution model alternately trains the generator G and the discriminator D through the data set S. After the model converges, the generator G is deployed behind the camera module of the ultra-small component sorting equipment, and the low-resolution image output by the camera module is reconstructed into a high-resolution image to improve the accuracy of the visual positioning module. The visual positioning module transmits the position of the component identified and obtained on the high-resolution image to the sorting device controller to complete the sorting of ultra-small components. Embodiment 2

[0098] In this example, the present invention is different from the first embodiment in that, in this embodiment, it is used for feature enhancement of small size defects, and its principle diagram is as follows: Figure 8As shown in the figure. The super-resolution model alternately trains the generator G and the discriminator D through the data set S. After the model converges, the generator G is deployed on the camera module of the AOI equipment, and the low-resolution image output by the camera module is reconstructed into a high-resolution image to improve the accuracy of defect visual detection. The visual inspection module will identify the acquired defect features on the high-resolution image and output the defect visual detection results.

[0099] The present invention comprehensively considers the similarity between the reconstructed high-resolution image and the current image and the image in the data set, and the obtained model can be used for high-resolution image reconstruction in the ultra-small component sorting scenario, or for high-resolution image reconstruction of defect detection in the AOI scenario, to achieve defect feature enhancement. That is, the present invention reconstructs the collected low-resolution image into a high-resolution image, and reconstructs and obtains a high-quality high-resolution image without increasing the hardware cost, thereby improving the positioning accuracy of the visual module, improving the sorting success rate of the ultra-small component sorting equipment, or improving the positioning accuracy of the visual module, and improving the detection success rate of small-size defects of the AOI equipment; in addition, the present invention can achieve plug-and-play, that is, only the model needs to be deployed between the camera module and the visual positioning module to improve the positioning accuracy of the visual detection module or improve the AOI detection accuracy of the visual detection module.

[0100] Although the embodiments of the present invention are described with practical solutions, they do not constitute limitations on the meaning of the present invention. For those skilled in the art, it is obvious to modify the implementation scheme and combine it with other solutions based on this description.

Claims

1. A method for realizing super-resolution in an automated device, characterized in that: The method comprises the following steps: Step S1: Create a sample dataset S, collect low-resolution images of the equipment working area and the corresponding high-resolution images in pairs, S = {(LR i , HR i )}, where LR i and HR i denote the low-resolution image and the corresponding high-resolution image of the i-th ultra-small component in the data set S respectively; Step S2: construct a generator model, the generator G is composed of multiple convolution modules, nonlinear modules and upsampling modules, and is responsible for reconstructing the input low-resolution image into a corresponding high-resolution image; the generator G includes a first convolution layer, a first residual block, a second residual block, a second convolution layer, a first BN layer, a third residual block, a first upsampling block, a second upsampling block and a third convolution layer; after the low-resolution image is input from the first convolution layer, it passes through the first residual block, the second residual block, the second convolution layer, the first BN layer, the first residual layer, the first upsampling block, the second upsampling block and the third convolution layer in sequence, and finally outputs a high-resolution image from the third convolution layer; the signal output end of the first convolution layer is divided into two paths, one path enters the first residual block, and the other path enters the first residual layer; Step S3: Construct a discriminator model. The discriminator D consists of multiple convolution modules, nonlinear modules, and downsampling modules. It is responsible for determining whether the input image is a reconstructed image by the generator or a real high-resolution image during the training process. The discriminator D includes convolution layer I, convolution layer II, BN layer I, convolution layer III, BN layer II, and fully connected layer in sequence; Step S4: Create a VGG model, which extracts image features for calculating model training loss; Step S5: training the model, setting the optimizers of the generator G and the discriminator D to Adam and SGD respectively, setting the learning rate scheduling scheme to cosine simulated annealing, and training the generator G and the discriminator D alternately; Step S6: deploying the trained model between the camera module and the visual positioning module for accurate sorting of ultra-small components or for visual inspection of small-sized defects; In step S5, each training cycle first sets the learning rate of the optimizer, and then performs alternating training of the generator and the discriminator. The specific content of one training is as follows: ①Train the discriminator: First, take the real high-resolution image HR i Pass it into the discriminator D to get D_real_loss, and then pass the real low-resolution image LR i After passing the image into the generator, it is passed into the discriminator to get D_fake_loss, and finally D_train_loss is obtained as: D_train_loss = D_real_loss + D_fake_loss; ② Train the generator: pass the real image into the generator to get the result, and obtain the image error image_loss through the mean square error between the generator output image and the real image; then pass the generated image into the discriminator to calculate adversarialloss, that is, the probability that the discriminator will identify the generated image as a real image; finally, pass the generated image and the real image into the VGG19 model and calculate the mean square error to obtain perception_loss; finally, calculate the generator's G_train_loss as: G_train_loss = image_loss + 10 -3 ×adversarial loss + 2×10 -6 ×perception_loss; Among them, image_loss and perception_loss reflect the pixel and texture differences between the reconstructed image and the real high-resolution image, respectively.

2. The method for realizing super-resolution in an automated device according to claim 1, characterized in that: The equipment in step S1 is an ultra-small component sorting equipment, or an AOI optical inspection equipment for small-size defect detection.

3. The method for realizing super-resolution in an automated device according to claim 1, characterized in that: The modules of generator G are: The first convolutional layer: the number of input channels is 3, the number of output channels is 64, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used. This layer converts the 3-channel color image channel of the input image into 64 channels; First residual block: the number of input channels is 64; Second residual block: the number of input channels is 64; Second convolutional layer: The number of input channels and output channels are both 64, the convolution kernel size is 3×3, the padding is set to 1, and the PReLU activation function is used; The first BN layer: normalizes the convolutional data and sets the number of features to 64; The first residual layer: performs pixel-level addition, with 64 input channels; First upsampling layer: the number of input channels is 64; Second upsampling layer: the number of input channels is 64; The third convolutional layer: the number of input channels is 64, the number of output channels is 3, the convolution kernel size is 9×9, the padding is set to 4, and the PReLU activation function is used; this layer converts the feature map into 3 channels to obtain the final high-resolution image.

4. The method for realizing super-resolution in an automated device according to claim 3, characterized in that: The first residual block and the second residual block each include a convolutional layer 1, a BN layer 1, a convolutional layer 2, a BN layer 2 and a residual layer 1 in sequence, wherein: Convolutional layer 1: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PreLU activation function is used; BN layer 1: normalize the convolutional data, and the number of features is channels; Convolutional layer 2: The convolution kernel size is 3×3, padding is set to 1, the number of input and output channels is channels, and the PReLU activation function is used; BN layer 2: normalize the convolutional data, and the number of features is channels; Residual layer 1: Residual learning is implemented by adding the input to the convolutional and batch normalized output.

5. The method for realizing super-resolution in an automated device according to claim 4, characterized in that: The first upsampling layer and the second upsampling layer both include a convolutional layer A and a Pixel Shuffler layer, where: Convolutional layer A: The convolution kernel size is 3×3, padding is set to 1, the number of input channels is in_channels, the number of output channels is in_channels×up_scale2, and the PreLU activation function is used to expand each channel in the input feature map to in_channels×up_scale2 channels. This operation will prepare data for the subsequent Pixel Shuffler layer; Pixel Shuffler layer: performs an upsampling operation, rearranging the pixels in the feature map to increase the resolution.

6. The method for realizing super-resolution in an automated device according to claim 1, characterized in that: In step S3, Convolutional layer I: The number of input channels is 3, the number of output channels is 64, the convolution kernel size is 3×3, the padding is set to 1, the Leaky ReLU activation function is used, and the slope is set to 0.2; Convolutional layer II: The number of input channels is 64, the number of output channels is 64, the convolution kernel size is 3×3, the stride is set to 2, and the padding is set to 1; the Leaky ReLU activation function is used, and the slope is set to 0.2; BN layer I: the number of features is set to 64; Convolutional layer III: the number of input channels is 64, the number of output channels is 128, the convolution kernel size is 3×3, the stride is set to 1, and the padding is set to 1; BN layer: the number of features is set to 128; the Leaky ReLU activation function is used, and the slope is set to 0.2; BN layer II: the number of features is set to 64; Fully connected layer: Use Conv2d, the number of input channels is 512, the number of output channels is set to 1024, the convolution kernel is set to 1×1, the sigmoid activation function is used, and the judgment result is output.

Images