Cloud rendering platform integrated picture super-resolution reconstruction method

Abstract

The application discloses a cloud rendering platform integrated picture super-resolution reconstruction method, which first trains a middleware for performing super-resolution reconstruction, and integrates the middleware to a deployment of a cloud rendering platform on a graphics card machine hardware; when a super-resolution lossless magnification task needs to be performed, the cloud rendering platform performs logical processing, calls an API interface of the middleware, the middleware requests to access a picture storage address of an original task, performs AI magnification processing after obtaining a low-resolution picture, and finally outputs a generated equal-proportion high-definition magnified rendering product picture to the cloud rendering platform. The application can generate an equal-proportion high-definition magnified rendering product picture in the cloud by integrating the picture super-resolution reconstruction technology on the cloud rendering platform. The generated equal-proportion high-definition magnified rendering product picture meets the demand of users for high-quality images, improves user experience, and reduces rendering cost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud rendering, and particularly relates to a cloud rendering platform integrated picture super-resolution reconstruction method. BACKGROUND

[0002] The rendered effect picture of a user using a cloud rendering platform may not meet the requirements in terms of resolution due to external reasons such as rendering scene, rendering setting, and rendering machine configuration. Therefore, re-rendering will consume double time and cost.

[0003] Therefore, the prior art has defects and needs to be improved. SUMMARY

[0004] The present application provides a cloud rendering platform integrated picture super-resolution reconstruction method, which generates a high-definition enlarged rendering product picture in the cloud by using super-resolution reconstruction technology for a low-resolution picture, so as to reduce rendering cost and improve efficiency.

[0005] The technical scheme of the present application is as follows: a cloud rendering platform integrated picture super-resolution reconstruction method is provided, which comprises the following steps.

[0006] S1: Collect a large number of high-resolution images as a training data set, and these images should contain details under different scenes, objects, and different lighting conditions.

[0007] S2: Use a deep convolutional neural network (DCNN) based on a generative adversarial network (GAN), which has multiple levels and different convolution kernels, so as to capture different details in the image.

[0008] S3: Use the pictures of the training data set for training, define a training function, a loss training function, and an optimizer, iterate the data set by loop and call the training function for GAN training, and output the training progress at the same time, and constantly adjust the loss function, the optimizer, and the hyperparameters until the required training effect is obtained. The required training result is, for example, the following three kinds:

[0009] Quality of generated images: the generated images should have high resolution and clarity, and be visually similar to real images. This is one of the most intuitive criteria for evaluating the performance of GAN.

[0010] Diversity of generated images: the generated images should cover the diversity in the training data set, and avoid mode collapse, that is, the generator starts to repeatedly generate a few modes of images.

[0011] Stability of training: The training process of GAN should be stable, avoiding instability in the training process, such as the discriminator being too strong causing the generator to fail to learn, or the generator being too strong causing the discriminator to always output high confidence false results.

[0012] S4: Use the deconvolution function provided by the deep learning library TensorFlow to implement upsampling. Through upsampling operation, the low-resolution image can be enlarged to the target size.

[0013] S5: Adjust the size, stride, and channel number parameters of the convolution kernel to capture local and global features of the image for feature reconstruction.

[0014] S6: Introduce denoising and smoothing processing in the process of generating images to improve the quality of enlarged images.

[0015] S7: After training and evaluation, it is packaged into a middleware and deployed on GPU machine hardware, and applied to actual image enlargement tasks to enlarge the input low-resolution image.

[0016] S8: For the rendered task, submit a super-resolution lossless enlargement task based on the original task and input the enlargement factor, such as 2x, 4x, etc.

[0017] S9: For the super-resolution lossless enlargement task, add a special marker to isolate it from other ordinary rendering tasks, and isolate the rendering node machine for scheduling; specify the machine for scheduling the super-resolution lossless enlargement task as the GPU machine where the middleware is deployed.

[0018] S10: After submitting the lossless enlargement task, the cloud rendering platform performs logical processing, calls the middleware API interface, and requests access to the original task picture storage address. After obtaining the low-resolution picture, it performs AI enlargement processing, and finally outputs the generated equal-proportion high-definition rendering product picture to the cloud rendering platform for storage, display, or transmission. Subsequent operations.

[0019] S11: Display the enlarged picture in the user interface and provide it for the user to download.

[0020] Further, the high-resolution image is obtained from a public dataset or the Internet or self-collected.

[0021] Further, use the Python image processing library OpenCV to load and prepare the image dataset to ensure the consistency, quality, and size resolution of the image data.

[0022] Further, in step S5, the process involves extracting the style features of one picture and applying them to the content features of another picture; specifically as follows.

[0023] S51: Capture local features, i.e., capture detailed features;

[0024] Kernel size: Start with smaller convolution kernels, such as 3x3 or 5x5, as these can capture local details and edge information of the image, which is crucial for replicating texture and details in style features;

[0025] Stride: Use a stride of 1 to preserve as much information as possible, as style features tend to be very fine-grained;

[0026] Padding: Use "same" padding to maintain the size of the feature maps and avoid losing important information at the edges;

[0027] Number of channels: Use fewer channels in the initial layers and gradually increase them as the network deepens to capture more complex features.

[0028] S52: Capture global features, i.e., capture style features;

[0029] Kernel size: Use larger convolution kernels, such as 7x7, as the network levels deepen, to capture more extensive features, which is crucial for style feature extraction in style transfer;

[0030] Stride: Use a stride of 1 to preserve more information;

[0031] Padding: Continue using "same" padding;

[0032] Number of channels: Increase the number of channels as the network levels deepen to capture more abstract style features.

[0033] S53: Feature reconstruction, operated by the decoder;

[0034] Kernel size: In the decoder part, use smaller convolution kernels again to finely reconstruct the image content;

[0035] Stride: Use a stride of 1 to accurately reconstruct the feature maps;

[0036] Padding: Use "same" padding to maintain the size of the feature maps;

[0037] Number of channels: In the decoder, gradually reduce the number of channels until the output layer matches the number of channels of the target image;

[0038] Up-sampling: In the decoder, use up-sampling (such as transposed convolution) to increase the size of the feature maps while reducing the number of channels to reconstruct the details of the image.

[0039] Through such a network structure and parameter adjustment, style features can be effectively extracted from a style image, and the features are applied to a content image, so that image style migration is realized; in this process, different levels of CNN capture different features from local to global, and the features are fused into the final output image through a feature reconstruction step.

[0040] Further, a loss function is set to evaluate the effect; in style migration, two kinds of loss functions are defined: content loss and style loss; the content loss ensures that the output image retains the content features of the original image, and the style loss ensures that the style of the output image matches the style of the reference image.

[0041] With the above scheme, the application provides a cloud rendering platform integrated picture super-resolution reconstruction method, which can generate a proportional high-definition enlarged rendering product picture in the cloud by integrating the picture super-resolution reconstruction technology on the cloud rendering platform. The generated proportional high-definition enlarged rendering product picture meets the user's demand for high-quality images, improves the user experience and reduces the rendering cost. DETAILED DESCRIPTION

[0042] The application will be described in detail below in combination with specific embodiments.

[0043] The application provides a cloud rendering platform integrated picture super-resolution reconstruction method, comprising the following steps.

[0044] S1: Collect a large number of high-resolution images as training data sets, which should contain details under different scenes, objects and different lighting conditions. High-resolution images are obtained from public data sets or the Internet or self-collection. Use the Python image processing library OpenCV to load and prepare the image data set, to ensure the consistency, quality and size resolution of the image data.

[0045] S2: Use a deep convolutional neural network (DCNN) based on a generative adversarial network (GAN), which has multiple levels and different convolution kernels to capture different details in the image.

[0046] S3: Use the pictures of the training data set for training, define the training function, the loss training function and the optimizer, iterate the data set through a loop and call the training function for GAN training, while outputting the training progress, and continuously adjust the loss function, the optimizer and the hyperparameters until the required training effect is obtained.

[0047] S4: Implement upsampling using the deconvolution function provided by the deep learning library TensorFlow. Through the upsampling operation, the low-resolution image can be enlarged to the target size.

[0048] S5: Feature reconstruction of the picture by adjusting the size, stride, and number of channels of the convolutional kernel to capture both local and global features of the image.

[0049] S6: Introduce denoising and smoothing during the generation of the image to improve the quality of the upscaled image.

[0050] S7: After training and evaluation, package it as a middleware and deploy it on GPU machine hardware for actual image upsizing tasks.

[0051] S8: For the rendered task, submit a super-resolution lossless upsizing task based on the original task and input the upsizing factor, such as 2x, 4x, etc.

[0052] S9: For the super-resolution lossless upsizing task, add a special marker to isolate it from other ordinary rendering tasks and isolate the rendering node machine for scheduling; specify the GPU machine (GPU machine) with middleware deployed for super-resolution lossless upsizing task scheduling.

[0053] S10: After submitting the lossless upsizing task, the cloud rendering platform performs logical processing, calls the middleware API interface, and requests access to the original task's picture storage address. After obtaining the low-resolution picture, it performs AI upsizing processing and finally outputs the generated equal-proportion high-definition rendering product picture to the cloud rendering platform for storage, display, or transmission for subsequent operations.

[0054] S11: Display the upsized picture in the user interface and allow the user to download it.

[0055] In this embodiment, in step S5, the process involves extracting the style features of one picture and applying them to the content features of another picture; the details are as follows.

[0056] S51: Capture local features, i.e., capture detailed features;

[0057] Convolution kernel size: Start with a small convolution kernel, such as 3x3 or 5x5, as these can capture local details and edge information of the image, which is crucial for replicating textures and details in the style features;

[0058] Stride: Use a stride of 1 to preserve as much information as possible, as style features are often very fine;

[0059] Padding: Use "same" padding to maintain the size of the feature map and avoid losing important information at the edges;

[0060] Number of channels: Use fewer channels in the initial layers and gradually increase them as the network deepens to capture more complex features.

[0061] S52: Capture global features, i.e. capture style features;

[0062] Convolution kernel size: As the network level deepens, larger convolution kernels such as 7x7 are used to capture more extensive features, which is very important for style feature extraction in style transfer;

[0063] Stride: Use a stride of 1 to retain more information;

[0064] Padding: Continue to use "same" padding;

[0065] Number of channels: As the network level deepens, increase the number of channels to capture more abstract style features.

[0066] S53: Feature reconstruction, through the operation of the decoder;

[0067] Convolution kernel size: In the decoder part, smaller convolution kernels are used again to finely reconstruct the image content;

[0068] Stride: Use a stride of 1 to accurately reconstruct the feature map;

[0069] Padding: Use "same" padding to maintain the size of the feature map;

[0070] Number of channels: In the decoder, the number of channels gradually decreases until the number of channels of the output layer matches the number of channels of the target image;

[0071] Up-sampling: In the decoder, up-sampling (such as transposed convolution) is used to increase the size of the feature map while reducing the number of channels to reconstruct the details of the image.

[0072] Through such network structure and parameter adjustment, style features can be effectively extracted from style images and applied to content images, thus realizing image style transfer; in this process, different levels of CNN capture different features from local to global, and these features are fused into the final output image through the feature reconstruction step.

[0073] In this embodiment, loss functions are set to evaluate the effect; in style transfer, two kinds of loss functions are defined: content loss and style loss; content loss ensures that the output image retains the content features of the original image, while style loss ensures that the style of the output image matches the style of the reference image.

[0074] In summary, the application provides a cloud rendering platform integrated picture super-resolution reconstruction method, by integrating the picture super-resolution reconstruction technology on the cloud rendering platform, the rendering product picture of equal proportion high-definition magnification can be generated in the cloud. The generated rendering product picture of equal proportion high-definition magnification meets the demand of users for high-quality images, improves user experience and reduces rendering cost.

[0075] The above is only a preferred embodiment of the application, and is not used to limit the application, and any modification, equivalent replacement and improvement made within the spirit and principle of the application should be included in the protection scope of the application.

Claims

1. A cloud rendering platform integrated picture super-resolution reconstruction method, characterized in that, The steps include: S1: Collect a large number of high-resolution images as a training dataset, which should contain details of different scenes, objects and different lighting conditions; S2: Use a deep convolutional neural network based on a generative adversarial network with multiple levels and different convolution kernels to capture different details in the image; S3: Use the pictures of the training dataset for training, define the training function, loss training function and optimizer, iterate the dataset through a loop and call the training function for GAN training, while outputting the training progress, and continuously adjust the loss function, optimizer and hyperparameters until the required training effect is obtained; S4: Implement upsampling using the deconvolution function provided by the deep learning library, which can enlarge the low-resolution image to the target size through upsampling operation; S5: Adjust the size, stride and channel number parameters of the convolution kernel to capture local and global features of the image for feature reconstruction of the picture; S6: Introduce denoising and smoothing processing in the process of generating images to improve the quality of enlarged images; S7: After training and evaluation, it is packaged into a middleware and deployed on a GPU machine hardware to be applied to actual image enlargement tasks for input low-resolution image enlargement processing; S8: For the rendered task, submit a super-resolution lossless enlargement task based on the original task and input the enlargement factor; S9: For the super-resolution lossless enlargement task, add a special mark to isolate it from other ordinary rendering tasks, and isolate the rendering node machine for scheduling; specify the GPU machine deployed with the middleware as the machine for scheduling the super-resolution lossless enlargement task; S10: After submitting the lossless enlargement task, the cloud rendering platform performs logical processing, calls the api interface of the middleware, requests access to the picture storage address of the original task, obtains the low-resolution picture and performs AI enlargement processing, and finally outputs the generated equal-proportion high-definition rendering product picture to the cloud rendering platform; S11: Display the enlarged picture in the user interface and provide it for the user to download; In step S5, the process involves extracting the style features of a picture and applying them to the content features of another picture; the details are as follows: S51: Capture local features, i.e. capture detail features; Convolution kernel size: start with a small convolution kernel to capture local details and edge information of the image; Stride: use a stride of 1; Padding: use "same" padding to maintain the size of the feature map and avoid losing important information at the edges; Channel number: use fewer channels in the initial layers and gradually increase them as the network deepens to capture more complex features; S52: Capture global features, i.e. capture style features; Convolution kernel size: use larger convolution kernels as the network level deepens to capture more extensive features; Stride: use a stride of 1 to retain more information; Padding: continue to use "same" padding; Channel number: increase the number of channels as the network level deepens to capture more abstract style features; S53: Feature reconstruction through decoder operation; Convolution kernel size: In the decoder part, smaller convolution kernels are used again to finely reconstruct the image content; Stride: Use a stride of 1 to accurately reconstruct the feature map; Padding: Use "same" padding to maintain the size of the feature map; Channel number: In the decoder, the number of channels gradually decreases until the output layer matches the number of channels of the target image; Up-sampling: In the decoder, up-sampling is used to increase the size of the feature map while reducing the number of channels to reconstruct the details of the image.

2. The cloud rendering platform integrated picture super-resolution reconstruction method of claim 1, wherein, High-resolution images are obtained from public datasets or the Internet or self-collection.

3. The cloud rendering platform integrated picture super-resolution reconstruction method of claim 1, wherein, Use the Python image processing library OpenCV to load and prepare the image dataset, ensuring consistency, quality, and size resolution of the image data.

4. The cloud rendering platform integrated picture super-resolution reconstruction method of claim 1, wherein, Set up loss functions to evaluate the effect; in style transfer, define two loss functions: content loss and style loss; content loss ensures that the output image retains the content features of the original image, while style loss ensures that the style of the output image matches the style of the reference image.