Data processing method and system, storage medium and computing device

Abstract

The present application discloses a data processing method and system, a storage medium and a computing device. The method includes: obtaining a first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, wherein the scales of the multiple generation networks are different. The present application solves the technical problem that the data processing method in the related art has a poor processing effect when restoring an image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a data processing method and system, a storage medium, and a computing device. Background Art

[0002] At present, for blurred and low-quality images, image restoration can be achieved in the following ways: face image generation algorithm and super-resolution algorithm. Among them, the face generation algorithm uses an adversarial generative network to synthesize high-definition face images, and the super-resolution algorithm usually fills in details based on the statistical characteristics of local image blocks (patches).

[0003] However, it is difficult for face generation algorithms to effectively control the content attributes of the output faces, and the attributes of the high-definition faces are difficult to keep consistent with the input. The super-resolution algorithm does not model the structural properties of the face, resulting in false textures often appearing on the faces in the synthesized video, affecting the quality.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a data processing method and system, a storage medium and a computing device to at least solve the technical problem that the data processing method in the related art has a poor processing effect during image restoration.

[0006] According to one aspect of an embodiment of the present application, a data processing method is provided, including: acquiring a first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0007] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: acquiring a first image containing a target object; downsampling the first image multiple times to obtain multiple downsampled images; processing the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0008] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: receiving an input first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0009] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: receiving an input first image containing a target object, and multiple downsampling rates; downsampling the first image using multiple downsampling rates to obtain multiple downsampled images; processing the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0010] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from that of the first image sample; using the multiple groups of training samples to sequentially train the encoding network and multiple generation networks in a processing model, wherein the processing model is used to input the first image sample into the encoding network to obtain a feature map of the first image sample, and input multiple downsampled first image samples and feature maps into multiple generation networks to obtain a second image sample, and the scales of the multiple generation networks are different; obtaining a first image containing a target object; and processing the first image using the processing model to obtain a second image containing the target object.

[0011] According to another aspect of an embodiment of the present application, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned data processing method.

[0012] According to another aspect of an embodiment of the present application, a computing device is further provided, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the above-mentioned data processing method is performed when the program is run.

[0013] According to another aspect of an embodiment of the present application, a data processing system is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: acquiring a first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0014] In an embodiment of the present application, after acquiring the first image, the encoding network in the processing model can be used to encode the first image, and multiple generation networks can be used to upsample the feature map output by the encoding network based on the downsampled first image to obtain a second image containing the target object, thereby achieving the purpose of image restoration of blurred and low-quality images. It is easy to notice that the processing model uses multiple generation networks connected in sequence, the scales of the multiple generation networks are different, and the generation network is processed based on the downsampled first image, thereby achieving the purpose of detail enhancement for the face area in the low-quality video and obtaining a more realistic high-definition character video, achieving the technical effect of improving the detail quality of the image and improving the processing effect, thereby solving the technical problem of poor processing effect of the data processing method in the related technology during image restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a data processing method according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of a first data processing method according to an embodiment of the present application;

[0018] Figure 3 is a flow chart of an optional data processing method according to an embodiment of the present application;

[0019] Figure 4 is an optional comparison diagram of an input image and an output image according to an embodiment of the present application;

[0020] Figure 5 is a flow chart of a second data processing method according to an embodiment of the present application;

[0021] Figure 6 is a flowchart of a third data processing method according to an embodiment of the present application;

[0022] Figure 7 is a schematic diagram of an interactive interface according to an embodiment of the present application;

[0023] Figure 8 is a flowchart of a fourth data processing method according to an embodiment of the present application;

[0024] Fig. 9 is a schematic diagram of another interactive interface according to an embodiment of the present application;

[0025] Fig.10 is a schematic diagram of a first data processing device according to an embodiment of the present application;

[0026] Fig.11 is a schematic diagram of a second data processing device according to an embodiment of the present application;

[0027] Fig.12 is a schematic diagram of a third data processing device according to an embodiment of the present application;

[0028] Fig.13 is a schematic diagram of a fourth data processing device according to an embodiment of the present application;

[0029] Fig.14 is a flowchart of a fifth data processing method according to an embodiment of the present application; and

[0030] Fig.15 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0034] Progressive network: a network structure that uses a multi-layer structure to synthesize images / videos and other content from coarse to fine.

[0035] Example 1

[0036] According to an embodiment of the present application, a data processing method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0038] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As with the processors involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned data processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0042] It should be noted that, in some optional embodiments, the above Figure 1 The computer device (or mobile device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the above-described computer device (or mobile device).

[0043] It should be noted that, in some embodiments, the above Figure 1 The computer device (or mobile device) shown has a touch display (also called a "touch screen" or "touch display screen"). In some embodiments, the above Figure 1 The computer device (or mobile device) shown has a graphical user interface (GUI), and a user can perform human-computer interaction with the GUI through finger contacts and / or gestures on the touch-sensitive surface. The human-computer interaction functions here optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interface, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for performing the above-mentioned human-computer interaction functions are configured / stored in a computer program product executable by one or more processors or a readable storage medium.

[0044] Under the above operating environment, this application provides Figure 2 The data processing method shown. Figure 2 is a flow chart of a first data processing method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0045] Step S202, acquiring a first image containing a target object;

[0046] The target object in the above steps may be an object specified by the user. For example, in the field of image restoration, the first image may be a blurred, low-quality image obtained by photographing the target object. In the embodiment of the present application, the image may be a face image uploaded by the user. The image is blurred or of low quality and needs to be converted. Another example is in the field of design, such as the design of a product detail page in an online shopping platform. The first image may be a blurred, low-quality image obtained by photographing the product. In the embodiment of the present application, the image may be a product image uploaded by the user. The image is blurred or of low quality and needs to be converted.

[0047] Step S204, using the processing model to process the first image to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0048] In the field of image restoration, the second image in the above step can be a clear, high-quality image, on this basis, the scales of multiple generation networks can be increased successively.

[0049] The encoding network in the above steps can adopt a convolutional neural network, which can perform feature encoding on the input blurred, low-quality image to obtain a feature map of the blurred, low-quality image. In the above embodiment of the present application, the resolution of 64*64 can be 64*64, but is not limited to this, and can also be other resolutions.

[0050] The multiple generation networks in the above steps can adopt a progressive network structure, and can be generated from fuzzy, low-quality images by increasing the number of network layers (i.e., the number of generation networks), and gradually improve the quality of detail generation, thereby obtaining a clear, high-quality image. In an embodiment of the present application, the generation network can be composed of an adaptive feature normalization (Spatially-Adaptive Normalization, SPADE) layer based on a spatiotemporal attention mechanism, but is not limited thereto, and can also be replaced by other similar structures. In addition, the scale scaling ratio of two adjacent generation networks is not limited and can be determined according to actual needs. Moreover, the number of generation networks can be determined based on actual processing accuracy requirements.

[0051] In order to improve the quality of details in blurry and low-quality images and eliminate some false textures, the downsampled input image can be used as a spatial structure guide in multiple generative networks. As the scales of the multiple generative networks increase successively, the sampling rates of the downsampled first images input to the multiple generative networks increase successively, and the scaling ratio of the sampling rate can be determined based on the scaling ratio of the scale of the generative network.

[0052] For example, Figure 3 As shown, a processing model consisting of an encoder and a generator can be pre-built, and the generator part consists of multiple SPADE layers. The scale ratios of the multiple SPADE layers are 4*4, ..., 64*64, 128*128, 256*256 and 512*512, and the sampling rate ratios of the down-sampled images input to each SPADE layer are: 1 / 128, ..., 1 / 8, 1 / 4, 1 / 2 and 1 / 1.

[0053] In an optional embodiment, after obtaining a 512*512 blurred, low-quality image, the blurred, low-quality image is input to an encoder, which performs feature encoding on the blurred, low-quality image to obtain a 4*4 feature map, and the feature map is input to a generator, which uses the downsampled blurred, low-quality image as structural information to gradually improve the resolution of the feature map to obtain a 512*512 clear, high-quality image. The comparison results of the input blurred, low-quality image and the output clear, high-quality image are shown in Figure 1. Figure 4 shown.

[0054] For example, taking the live broadcast application scenario as an example, during the live broadcast, the client can upload the host's image to the server. The server calls the trained processing model to process the received image to obtain a clear, high-quality image, and provide the clear, high-quality image to users watching the live broadcast.

[0055] Based on the solution provided by the above-mentioned embodiment of the present application, after acquiring the first image, the encoding network in the processing model can be used to encode the first image, and multiple generation networks can be used to upsample the feature map output by the encoding network based on the downsampled first image to obtain a second image containing the target object, thereby achieving the purpose of image restoration of blurred and low-quality images. It is easy to notice that the processing model uses multiple generation networks connected in sequence, the scales of the multiple generation networks are different, and the generation network is processed based on the downsampled first image, thereby achieving the purpose of detail enhancement for the face area in the low-quality video and obtaining a more realistic high-definition character video, achieving the technical effect of improving the detail quality of the image and improving the processing effect, thereby solving the technical problem of poor processing effect of the data processing method in the related technology during image restoration.

[0056] In the above embodiment of the present application, the method also includes the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0057] Since it is difficult to obtain blurred, low-quality images and clear, high-quality images of the same object from the Internet, in an embodiment of the present application, a blurred, low-quality image can be artificially generated based on a clear, high-quality image, for example, by downsampling the clear, high-quality image, or by adding noise.

[0058] In an optional embodiment, in order to increase the stability of the processing model training, progressive training can be used to train the encoding network and multiple generation networks in sequence. In particular, after the training of one generation network is completed, the training process of the next generation network can be carried out to gradually improve the image resolution.

[0059] In the above embodiment of the present application, multiple groups of training samples are used to train multiple generation networks in sequence, including: determining a generation network to be trained among multiple generation networks, wherein the generation network before the generation network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generation network to be trained, and a fourth image output by the previous generation network connected to the generation network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; based on the discrimination result of the weighted image, determining whether the generation network to be trained has completed training, wherein, in the case that the generation network to be trained has completed training, determining the next generation network connected to the generation network to be trained as the generation network to be trained, until multiple generation networks have completed training.

[0060] In this embodiment, the synthesis of two adjacent generative networks can use weight α to fuse the results, that is, each generative network uses weight α, and the previous generative network uses weight (1-α). In addition, the training method of the adversarial generative network can be used to discriminate the input image through the identification network to determine whether it is an image input to the generative network or an image generated by the generative network, and then determine whether the generative network is trained. When the discrimination result output by the identification network is 0.5, it indicates that the identification network cannot determine whether the input image is generated by the generative network. At this time, it can be determined that the training of the generative network is completed, and the next generative network can be trained.

[0061] For example, Figure 3 As shown, the identification network can adopt the structure of a multi-layer convolutional neural network. For the first generative network, the training method of the adversarial generative network can be directly adopted, and multiple groups of training samples can be used for training. After the training is completed, the second generative network can be trained. The image I2 output by the second generative network (i.e., the third image mentioned above) can be obtained, and the image I1 output by the first generative network (i.e., the fourth image mentioned above) can be obtained, the weighted image I'=α*I2+(1-α)*I1, and then processed by the convolution layer corresponding to the second generative network, and the output result is input to the convolution layer corresponding to the first generative network, so as to obtain the discrimination result, and further based on the discrimination result, whether the second generative network is trained. Each subsequent generative network can be trained in turn in the above manner until all generative networks are trained.

[0062] In the above embodiment of the present application, obtaining multiple groups of training samples includes: obtaining multiple second image samples; adding noise to the second image samples to obtain first image samples.

[0063] In an optional embodiment, the second image sample may be obtained from an existing open source high-resolution face dataset, and the first image sample may be synthesized by manually adding noise to obtain a training sample pair.

[0064] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0065] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0066] Example 2

[0067] According to an embodiment of the present application, a data processing method is also provided.

[0068] Figure 5 is a flow chart of a second data processing method according to an embodiment of the present application. Figure 5 As shown, the method comprises the following steps:

[0069] Step S502, acquiring a first image containing a target object;

[0070] Step S504, downsampling the first image multiple times to obtain multiple downsampled images;

[0071] In an optional embodiment, the first image may be downsampled using different sampling rates to obtain multiple downsampled images.

[0072] Step S506, using the processing model to process the first image and multiple downsampled images to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain the second image, and the scales of the multiple generation networks are different.

[0073] In the above embodiment of the present application, the method also includes the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0074] In the above embodiment of the present application, multiple groups of training samples are used to train multiple generation networks in sequence, including: determining a generation network to be trained among multiple generation networks, wherein the generation network before the generation network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generation network to be trained, and a fourth image output by the previous generation network connected to the generation network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; based on the discrimination result of the weighted image, determining whether the generation network to be trained has completed training, wherein, in the case that the generation network to be trained has completed training, determining the next generation network connected to the generation network to be trained as the generation network to be trained, until multiple generation networks have completed training.

[0075] In the above embodiment of the present application, obtaining multiple groups of training samples includes: obtaining multiple second image samples; adding noise to the second image samples to obtain first image samples.

[0076] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0077] Example 3

[0078] According to an embodiment of the present application, a data processing method is also provided.

[0079] Figure 6 is a flow chart of a third data processing method according to an embodiment of the present application. Figure 6 As shown, the method comprises the following steps:

[0080] Step S602, receiving an input first image containing a target object;

[0081] In an optional embodiment, in order to facilitate users to upload blurry and low-quality images, a user may be provided with a Figure 7 In the interactive interface shown, the user can upload an image by clicking the "Upload Image" button or by directly dragging the image file into the dotted box.

[0082] Step S604, using the processing model to process the first image to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled first images and feature maps into multiple generation networks to obtain the second image, and the scales of the multiple generation networks are different;

[0083] Step S606: display the second image.

[0084] In an optional embodiment, if Figure 7 As shown, clear, high-quality images can be displayed in the display area of ​​the interactive interface for easy viewing by users.

[0085] In the above embodiment of the present application, the method also includes the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0086] In the above embodiment of the present application, multiple groups of training samples are used to train multiple generation networks in sequence, including: determining a generation network to be trained among multiple generation networks, wherein the generation network before the generation network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generation network to be trained, and a fourth image output by the previous generation network connected to the generation network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; based on the discrimination result of the weighted image, determining whether the generation network to be trained has completed training, wherein, in the case that the generation network to be trained has completed training, determining the next generation network connected to the generation network to be trained as the generation network to be trained, until multiple generation networks have completed training.

[0087] In the above embodiment of the present application, obtaining multiple groups of training samples includes: obtaining multiple second image samples; adding noise to the second image samples to obtain first image samples.

[0088] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0089] Example 4

[0090] According to an embodiment of the present application, a data processing method is also provided.

[0091] Figure 8 is a flow chart of a fourth data processing method according to an embodiment of the present application. Figure 8 As shown, the method comprises the following steps:

[0092] Step S802, receiving an input first image containing a target object and a plurality of downsampling rates;

[0093] In an optional embodiment, in order to facilitate users to upload blurry and low-quality images, a user may be provided with a Fig. 9 In the interactive interface shown, users can upload images by clicking the "Upload Image" button or directly dragging the image file to the dotted box. At the same time, users can enter multiple different sampling rates in the input area, and the order of multiple sampling rates is consistent with the order of multiple generated networks.

[0094] Step S804, downsampling the first image using multiple downsampling rates to obtain multiple downsampled images;

[0095] Step S806, using the processing model to process the first image and the multiple downsampled images to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different;

[0096] Step S808: display the second image.

[0097] In an optional embodiment, if Fig. 9 As shown, clear, high-quality images can be displayed in the display area of ​​the interactive interface for easy viewing by users.

[0098] In the above embodiment of the present application, the method also includes the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the resolution of the second image sample is greater than the resolution of the first image sample; using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0099] In the above embodiment of the present application, multiple groups of training samples are used to train multiple generation networks in sequence, including: determining a generation network to be trained among multiple generation networks, wherein the generation network before the generation network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generation network to be trained, and a fourth image output by the previous generation network connected to the generation network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; based on the discrimination result of the weighted image, determining whether the generation network to be trained has completed training, wherein, in the case that the generation network to be trained has completed training, determining the next generation network connected to the generation network to be trained as the generation network to be trained, until multiple generation networks have completed training.

[0100] In the above embodiment of the present application, obtaining multiple groups of training samples includes: obtaining multiple second image samples; adding noise to the second image samples to obtain first image samples.

[0101] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0102] Example 5

[0103] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.10 As shown, the device 1000 includes: a first acquisition module 1002 and a processing module 1004 .

[0104] Among them, the first acquisition module 1002 is used to acquire a first image containing a target object; the processing module 1004 is used to process the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0105] It should be noted that the first acquisition module 1002 and the processing module 1004 correspond to steps S202 to S204 in Example 1, and the examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0106] In the above embodiment of the present application, the device also includes: a second acquisition module and a training module.

[0107] Among them, the second acquisition module is used to obtain multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; the training module is used to use the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0108] In the above embodiments of the present application, the training module includes: a determination unit, an input unit, a processing unit and a determination unit.

[0109] The determining unit is used to determine a generating network to be trained among multiple generating networks, wherein the generating network before the generating network to be trained has completed training; the input unit is used to input multiple groups of training samples into the encoding network to obtain a third image output by the generating network to be trained, and a fourth image output by the previous generating network connected to the generating network to be trained; the processing unit is used to obtain a weighted sum of the third image and the fourth image to obtain a weighted image; the determining unit is used to determine whether the generating network to be trained has completed training based on a discrimination result of the weighted image, wherein, in the case that the generating network to be trained has completed training, the next generating network connected to the generating network to be trained is determined to be the generating network to be trained, until the multiple generating networks have completed training.

[0110] In the above embodiment of the present application, the second acquisition module includes: an acquisition unit and an adding unit.

[0111] The acquiring unit is used to acquire a plurality of second image samples; and the adding unit is used to add noise to the second image samples to obtain the first image samples.

[0112] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0113] Example 6

[0114] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.11 As shown, the device 1100 includes: a first acquisition module 1102 , a sampling module 1104 and a processing module 1106 .

[0115] Among them, the first acquisition module 1102 is used to acquire a first image containing a target object; the sampling module 1104 is used to downsample the first image multiple times to obtain multiple downsampled images; the processing module 1106 is used to process the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0116] It should be noted that the first acquisition module 1102, the sampling module 1104 and the processing module 1106 correspond to steps S502 to S506 in Example 2, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0117] In the above embodiment of the present application, the device also includes: a second acquisition module and a training module.

[0118] Among them, the second acquisition module is used to obtain multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; the training module is used to use the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0119] In the above embodiments of the present application, the training module includes: a determination unit, an input unit, a processing unit and a determination unit.

[0120] The determining unit is used to determine a generating network to be trained among multiple generating networks, wherein the generating network before the generating network to be trained has completed training; the input unit is used to input multiple groups of training samples into the encoding network to obtain a third image output by the generating network to be trained, and a fourth image output by the previous generating network connected to the generating network to be trained; the processing unit is used to obtain a weighted sum of the third image and the fourth image to obtain a weighted image; the determining unit is used to determine whether the generating network to be trained has completed training based on a discrimination result of the weighted image, wherein, in the case that the generating network to be trained has completed training, the next generating network connected to the generating network to be trained is determined to be the generating network to be trained, until the multiple generating networks have completed training.

[0121] In the above embodiment of the present application, the second acquisition module includes: an acquisition unit and an adding unit.

[0122] The acquiring unit is used to acquire a plurality of second image samples; and the adding unit is used to add noise to the second image samples to obtain the first image samples.

[0123] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0124] Example 7

[0125] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.12 As shown, the device 1200 includes: a receiving module 1202 , a processing module 1204 and a display module 1206 .

[0126] Among them, the receiving module 1202 is used to receive an input first image containing a target object; the processing module 1204 is used to process the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; the display module 1206 is used to display the second image.

[0127] It should be noted that the receiving module 1202, the processing module 1204 and the display module 1206 correspond to steps S602 to S606 in Example 3, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0128] In the above embodiments of the present application, the device further includes: an acquisition module and a training module.

[0129] Among them, the acquisition module is used to obtain multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; the training module is used to use the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0130] In the above embodiments of the present application, the training module includes: a determination unit, an input unit, a processing unit and a determination unit.

[0131] The determining unit is used to determine a generating network to be trained among multiple generating networks, wherein the generating network before the generating network to be trained has completed training; the input unit is used to input multiple groups of training samples into the encoding network to obtain a third image output by the generating network to be trained, and a fourth image output by the previous generating network connected to the generating network to be trained; the processing unit is used to obtain a weighted sum of the third image and the fourth image to obtain a weighted image; the determining unit is used to determine whether the generating network to be trained has completed training based on a discrimination result of the weighted image, wherein, in the case that the generating network to be trained has completed training, the next generating network connected to the generating network to be trained is determined to be the generating network to be trained, until the multiple generating networks have completed training.

[0132] In the above embodiments of the present application, the acquisition module includes: an acquisition unit and an adding unit.

[0133] The acquiring unit is used to acquire a plurality of second image samples; and the adding unit is used to add noise to the second image samples to obtain the first image samples.

[0134] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0135] Example 8

[0136] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Fig.13 As shown, the device 1300 includes: a receiving module 1302 , a sampling module 1304 , a processing module 1306 and a display module 1308 .

[0137] Among them, the receiving module 1302 is used to receive an input first image containing a target object, and multiple downsampling rates; the sampling module 1304 is used to downsample the first image using multiple downsampling rates to obtain multiple downsampled images; the processing module 1306 is used to process the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; the display module 1308 is used to display the second image.

[0138] It should be noted that the receiving module 1302, sampling module 1304, processing module 1306 and display module 1308 correspond to steps S802 to S808 in Example 4, and the examples and application scenarios implemented by the four modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules as part of the device can be run in the computer terminal 10 provided in Example 1.

[0139] In the above embodiments of the present application, the device further includes: an acquisition module and a training module.

[0140] Among them, the acquisition module is used to obtain multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; the training module is used to use the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0141] In the above embodiments of the present application, the training module includes: a determination unit, an input unit, a processing unit and a determination unit.

[0142] The determining unit is used to determine a generating network to be trained among multiple generating networks, wherein the generating network before the generating network to be trained has completed training; the input unit is used to input multiple groups of training samples into the encoding network to obtain a third image output by the generating network to be trained, and a fourth image output by the previous generating network connected to the generating network to be trained; the processing unit is used to obtain a weighted sum of the third image and the fourth image to obtain a weighted image; the determining unit is used to determine whether the generating network to be trained has completed training based on a discrimination result of the weighted image, wherein, in the case that the generating network to be trained has completed training, the next generating network connected to the generating network to be trained is determined to be the generating network to be trained, until the multiple generating networks have completed training.

[0143] In the above embodiments of the present application, the acquisition module includes: an acquisition unit and an adding unit.

[0144] The acquiring unit is used to acquire a plurality of second image samples; and the adding unit is used to add noise to the second image samples to obtain the first image samples.

[0145] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0146] Example 9

[0147] According to an embodiment of the present application, a data processing system is also provided, including:

[0148] Processor; and

[0149] The memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: obtaining a first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0150] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0151] Example 10

[0152] According to an embodiment of the present application, a data processing method is also provided.

[0153] Fig.14 is a flow chart of a fifth data processing method according to an embodiment of the present application. Fig.14 As shown, the method comprises the following steps:

[0154] Step S1402, obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample;

[0155] Step S1404, using multiple groups of training samples to train the encoding network and multiple generating networks in the processing model in sequence, wherein the processing model is used to input the first image sample into the encoding network to obtain a feature map of the first image sample, and input multiple downsampled first image samples and feature maps into multiple generating networks to obtain second image samples, and the scales of the multiple generating networks are different;

[0156] Step S1406, acquiring a first image containing a target object;

[0157] Step S1408: Process the first image using the processing model to obtain a second image containing the target object.

[0158] In the above embodiment of the present application, multiple groups of training samples are used to train multiple generation networks in sequence, including: determining a generation network to be trained among multiple generation networks, wherein the generation network before the generation network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generation network to be trained, and a fourth image output by the previous generation network connected to the generation network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; based on the discrimination result of the weighted image, determining whether the generation network to be trained has completed training, wherein, in the case that the generation network to be trained has completed training, determining the next generation network connected to the generation network to be trained as the generation network to be trained, until multiple generation networks have completed training.

[0159] In the above embodiment of the present application, obtaining multiple groups of training samples includes: obtaining multiple second image samples; adding noise to the second image samples to obtain first image samples.

[0160] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0161] Embodiment 11

[0162] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0163] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0164] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the data processing method: obtaining a first image containing a target object; using a processing model to process the first image to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0165] Optionally, Fig.15 is a structural block diagram of a computer terminal according to an embodiment of the present application. Fig.15 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1502 and a memory 1504.

[0166] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first image containing a target object; use a processing model to process the first image to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0168] Optionally, the processor may also execute the program code of the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from that of the first image sample; and using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0169] Optionally, the processor may also execute the following steps of program code: determining a generating network to be trained among multiple generating networks, wherein the generating network before the generating network to be trained has completed training; inputting multiple groups of training samples into the encoding network, obtaining a third image output by the generating network to be trained, and a fourth image output by a previous generating network connected to the generating network to be trained; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; and determining whether the generating network to be trained has completed training based on a judgment result of the weighted image, wherein, in the case that the generating network to be trained has completed training, determining the next generating network connected to the generating network to be trained as the generating network to be trained, until multiple generating networks have completed training.

[0170] Optionally, the processor may also execute program codes of the following steps: acquiring a plurality of second image samples; and adding noise to the second image samples to obtain first image samples.

[0171] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first image containing a target object; downsample the first image multiple times to obtain multiple downsampled images; use a processing model to process the first image and the multiple downsampled images to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0172] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving an input first image containing a target object; using a processing model to process the first image to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0173] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving an input first image containing a target object and multiple downsampling rates; downsampling the first image using multiple downsampling rates to obtain multiple downsampled images; processing the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into the encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0174] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from that of the first image sample; use the multiple groups of training samples to train the encoding network and multiple generation networks in the processing model in sequence, wherein the processing model is used to input the first image sample into the encoding network to obtain a feature map of the first image sample, and input multiple downsampled first image samples and feature maps into multiple generation networks to obtain a second image sample, and the scales of the multiple generation networks are different; obtain a first image containing a target object; and use the processing model to process the first image to obtain a second image containing the target object.

[0175] By using the embodiment of the present application, a data processing solution is provided. By using multiple generative networks connected in sequence in the processing model, the scales of the multiple generative networks are increased in sequence, and the generative networks are processed based on the first image after downsampling, thereby achieving the purpose of enhancing the details of the face area in the low-quality video and obtaining a more realistic high-definition character video, achieving the technical effect of improving the detail quality of the synthesized image and reducing false textures, thereby solving the technical problem of poor processing effect of the data processing method in the related art during image restoration.

[0176] It can be understood by those skilled in the art that Fig.15 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Fig.15 It does not limit the structure of the above electronic device. For example, the computer terminal A may also include Fig.15 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.15 Different configurations are shown.

[0177] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0178] Example 12

[0179] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided by the above embodiment.

[0180] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0181] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0182] Optionally, the storage medium is also configured to store program codes for executing the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; and using the multiple groups of training samples to train the encoding network and multiple generation networks in sequence.

[0183] Optionally, the storage medium is further configured to store a program code for executing the following steps: determining a to-be-trained generating network among a plurality of generating networks, wherein the generating network before the to-be-trained generating network has completed training; inputting a plurality of sets of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; obtaining a weighted sum of the third image and the fourth image to obtain a weighted image; and determining whether the to-be-trained generating network has completed training based on a discrimination result of the weighted image, wherein, in the case that the to-be-trained generating network has completed training, determining the next generating network connected to the to-be-trained generating network as the to-be-trained generating network, until the plurality of generating networks have completed training.

[0184] Optionally, the storage medium is further configured to store program codes for executing the following steps: acquiring a plurality of second image samples; and adding noise to the second image samples to obtain first image samples.

[0185] Optionally, in this embodiment, the storage medium is configured to store program code for executing the following steps: acquiring a first image containing a target object; downsampling the first image multiple times to obtain multiple downsampled images; processing the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from the clarity of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different.

[0186] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving an input first image containing a target object; processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, input multiple downsampled first images and feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0187] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving an input first image containing a target object, and multiple downsampling rates; downsampling the first image using multiple downsampling rates to obtain multiple downsampled images; processing the first image and the multiple downsampled images using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is different from that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, input the multiple downsampled images and the feature maps into multiple generation networks to obtain a second image, and the scales of the multiple generation networks are different; and displaying the second image.

[0188] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from that of the first image sample; using the multiple groups of training samples to train the encoding network and multiple generation networks in the processing model in sequence, wherein the processing model is used to input the first image sample into the encoding network to obtain a feature map of the first image sample, and input multiple downsampled first image samples and feature maps into multiple generation networks to obtain a second image sample, and the scales of the multiple generation networks are different; obtaining a first image containing a target object; and processing the first image using the processing model to obtain a second image containing the target object.

[0189] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0190] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0191] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0195] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, comprising: Acquire a first image containing a target object; The first image is processed by using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is greater than that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and the feature maps into multiple generation networks to obtain the second image, wherein the scales of the multiple generation networks are different, and the multiple generation networks adopt a progressive network structure.

2. The method according to claim 1, wherein: The method further comprises: Acquire multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; The multiple generation networks are trained in sequence using the multiple groups of training samples.

3. The method according to claim 2, wherein: Using the multiple groups of training samples to train the multiple generation networks in sequence includes: Determine a to-be-trained generating network among the plurality of generating networks, wherein the generating network preceding the to-be-trained generating network has completed training; Inputting the plurality of groups of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; Acquire a weighted sum of the third image and the fourth image to obtain a weighted image; Based on the discrimination result of the weighted image, it is determined whether the to-be-trained generating network has completed training, wherein, when the to-be-trained generating network has completed training, a next generating network connected to the to-be-trained generating network is determined to be the to-be-trained generating network, until the multiple generating networks have completed training.

4. The method according to claim 2, wherein: Get multiple sets of training samples, including: acquiring a plurality of second image samples; Noise is added to the second image sample to obtain the first image sample.

5. The method according to claim 1, wherein: The sampling rates of the downsampled first images input to the multiple generation networks increase sequentially.

6. A data processing method, comprising: Acquire a first image containing a target object; Downsampling the first image multiple times to obtain multiple downsampled images; The first image and the multiple downsampled images are processed by using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is greater than the clarity of the first image. The processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature map into multiple generation networks to obtain the second image. The scales of the multiple generation networks are different, and the multiple generation networks adopt a progressive network structure.

7. The method according to claim 6, wherein: The method further comprises: Acquire multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; The multiple generation networks are trained in sequence using the multiple groups of training samples.

8. The method according to claim 7, wherein: Using the multiple groups of training samples to train the multiple generation networks in sequence includes: Determine a to-be-trained generating network among the plurality of generating networks, wherein the generating network preceding the to-be-trained generating network has completed training; Inputting the plurality of groups of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; Acquire a weighted sum of the third image and the fourth image to obtain a weighted image; Based on the discrimination result of the weighted image, it is determined whether the to-be-trained generating network has completed training, wherein, when the to-be-trained generating network has completed training, a next generating network connected to the to-be-trained generating network is determined to be the to-be-trained generating network, until the multiple generating networks have completed training.

9. A data processing method, comprising: Receiving a first image including a target object; Processing the first image using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is greater than that of the first image, the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input a plurality of downsampled first images and the feature maps into a plurality of generating networks to obtain the second image, wherein the scales of the plurality of generating networks are different and the plurality of generating networks adopt a progressive network structure; The second image is displayed.

10. The method according to claim 9, wherein: The method further comprises: Acquire multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; The encoding network and the multiple generating networks are trained in sequence using the multiple groups of training samples.

11. The method according to claim 10, wherein: Using the multiple groups of training samples to train the multiple generation networks in sequence includes: Determine a to-be-trained generating network among the plurality of generating networks, wherein the generating network preceding the to-be-trained generating network has completed training; Inputting the plurality of groups of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; Acquire a weighted sum of the third image and the fourth image to obtain a weighted image; Based on the discrimination result of the weighted image, it is determined whether the to-be-trained generating network has completed training, wherein, when the to-be-trained generating network has completed training, a next generating network connected to the to-be-trained generating network is determined to be the to-be-trained generating network, until the multiple generating networks have completed training.

12. A data processing method, comprising: receiving a first image including a target object and a plurality of downsampling rates; Downsampling the first image using the multiple downsampling rates to obtain multiple downsampled images; Using a processing model to process the first image and the multiple downsampled images to obtain a second image containing the target object, wherein the clarity of the second image is greater than the clarity of the first image, the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input the multiple downsampled images and the feature map into multiple generation networks to obtain the second image, the multiple generation networks have different scales, and the multiple generation networks adopt a progressive network structure; The second image is displayed.

13. The method according to claim 12, wherein: The method further comprises: Acquire multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is different from the clarity of the first image sample; The multiple generation networks are trained in sequence using the multiple groups of training samples.

14. The method according to claim 13, wherein: Using the multiple groups of training samples to train the multiple generation networks in sequence includes: Determine a to-be-trained generating network among the plurality of generating networks, wherein the generating network preceding the to-be-trained generating network has completed training; Inputting the plurality of groups of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; Acquire a weighted sum of the third image and the fourth image to obtain a weighted image; Based on the discrimination result of the weighted image, it is determined whether the to-be-trained generating network has completed training, wherein, when the to-be-trained generating network has completed training, a next generating network connected to the to-be-trained generating network is determined to be the to-be-trained generating network, until the multiple generating networks have completed training.

15. A data processing method, comprising: Acquire multiple groups of training samples, wherein each group of training samples includes: a first image sample and a second image sample of the same object, and the clarity of the second image sample is greater than the clarity of the first image sample; Using multiple groups of training samples to train multiple generating networks in the processing model in sequence, wherein the processing model is used to input the first image sample into the encoding network to obtain a feature map of the first image sample, and input multiple downsampled first image samples and the feature map into the multiple generating networks to obtain the second image sample, the multiple generating networks have different scales, and the multiple generating networks adopt a progressive network structure; Acquire a first image containing a target object; The first image is processed using the processing model to obtain a second image containing the target object.

16. The method according to claim 15, wherein: Multiple sets of training samples are used to train multiple generating networks in the processing model in sequence, including: Determine a to-be-trained generating network among the plurality of generating networks, wherein the generating network preceding the to-be-trained generating network has completed training; Inputting the plurality of groups of training samples into the encoding network, obtaining a third image output by the to-be-trained generating network, and a fourth image output by a previous generating network connected to the to-be-trained generating network; Acquire a weighted sum of the third image and the fourth image to obtain a weighted image; Based on the discrimination result of the weighted image, it is determined whether the to-be-trained generating network has completed training, wherein, when the to-be-trained generating network has completed training, a next generating network connected to the to-be-trained generating network is determined to be the to-be-trained generating network, until the multiple generating networks have completed training.

17. The method according to claim 15, wherein: Get multiple sets of training samples, including: acquiring a plurality of second image samples; Noise is added to the second image sample to obtain the first image sample.

18. A storage medium comprising a stored program, wherein: When the program is running, the device where the storage medium is located is controlled to execute the data processing method described in any one of claims 1 to 17.

19. A computing device comprising: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the data processing method described in any one of claims 1 to 17 when running.

20. A data processing system comprising: processor; as well as A memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: acquiring a first image containing a target object; The first image is processed by using a processing model to obtain a second image containing the target object, wherein the clarity of the second image is greater than that of the first image, and the processing model is used to input the first image into an encoding network to obtain a feature map of the first image, and input multiple downsampled first images and the feature maps into multiple generation networks to obtain the second image, wherein the scales of the multiple generation networks are different, and the multiple generation networks adopt a progressive network structure.

Images