Image processing method, device, equipment and storage medium

Abstract

This application discloses an image processing method, apparatus, device, and storage medium, belonging to the field of artificial intelligence technology, and particularly relating to cloud computing, big data, or databases in the cloud technology field. The embodiments of this application can randomly select a processing method to degrade a high-resolution first sample image to obtain a low-resolution second sample image. This eliminates the need for manual processing, significantly reducing labor costs and improving the efficiency of acquiring low-resolution images. Furthermore, during training, a different processing method can be selected for degradation processing each time, ensuring that the trained target image processing model can perform good image processing on images processed by various methods, adapting to a wider range of image processing scenarios. Therefore, this target image processing model has good applicability.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, many scenarios require processing low-resolution images to obtain high-resolution images. Artificial intelligence technology allows image processing models to be trained using sample images, resulting in a target image processing model.

[0003] Currently, image processing methods typically involve acquiring high-resolution and low-resolution sample images to train image processing models. This requires a large number of sample images, which need to be processed by technicians using image processing applications. The acquisition of sample images is labor-intensive, resulting in high labor costs and low acquisition efficiency. Consequently, the efficiency of image processing methods is low. Summary of the Invention

[0004] This application provides an image processing method, apparatus, device, and storage medium, which can improve image processing efficiency and the applicability of the target image processing model. The technical solution is as follows:

[0005] On the one hand, an image processing method is provided, the method comprising:

[0006] Obtain the first sample image;

[0007] From at least two processing methods, one processing method is randomly selected to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image;

[0008] Based on the image processing model, super-resolution image processing is performed on the second sample image to obtain a third sample image, wherein the resolution of the third sample image is greater than that of the second sample image;

[0009] Based on the first sample image and the third sample image, the target loss value is obtained;

[0010] Based on the target loss value, the model parameters of the image processing model are updated;

[0011] Continue executing the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stop to obtain the target image processing model.

[0012] On one hand, an image processing apparatus is provided, the apparatus comprising:

[0013] The acquisition module is used to acquire the first sample image;

[0014] The degradation processing module is used to randomly select one processing method from at least two processing methods to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image;

[0015] The super-resolution processing module is used to perform super-resolution image processing on the second sample image based on the image processing model to obtain a third sample image, wherein the resolution of the third sample image is greater than the resolution of the second sample image.

[0016] The loss value acquisition module is used to acquire a target loss value based on the first sample image and the third sample image;

[0017] An update module is used to update the model parameters of the image processing model based on the target loss value;

[0018] The device continues to execute the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stops to obtain the target image processing model.

[0019] In some embodiments, the at least two processing methods include at least two of image blurring, adding image noise, image filtering, and image compression;

[0020] The degradation processing module is used for any of the following:

[0021] The first sample image is blurred to obtain the second sample image;

[0022] Add image noise to the first sample image to obtain the second sample image;

[0023] The first sample image is filtered to obtain the second sample image;

[0024] The first sample image is compressed to obtain the second sample image.

[0025] In some embodiments, the degradation processing module is used to randomly select a degradation model from at least two degradation models, and process the first sample image based on the degradation model and the degradation parameters corresponding to the degradation model.

[0026] In some embodiments, the degradation processing module is used for:

[0027] From at least two degradation parameters corresponding to the degradation model, randomly select one degradation parameter;

[0028] The first sample image is processed based on the degradation model and the randomly selected degradation parameters.

[0029] In some embodiments, the super-resolution processing module is used to input the second sample image into the image processing model, and at least two residual dense modules in the image processing model perform super-resolution image processing on the second sample image to obtain at least two residual images. Based on the at least two residual images and the second sample image, a third sample image is obtained and output; wherein, the at least two residual images are obtained by performing super-resolution image processing based on a different number of residual dense modules.

[0030] In some embodiments, the model parameters include the weights of at least two residual images;

[0031] The super-resolution processing module is used for:

[0032] The at least two residual images are fused with the second sample image to obtain at least two candidate third sample images;

[0033] Based on the weights of the at least two residual images, the at least two candidate third sample images are weighted to obtain the third sample image, and the third sample image is output.

[0034] In some embodiments, the residual dense module includes at least two convolutional layers connected in series with dense links.

[0035] In some embodiments, the loss value acquisition module is used for:

[0036] Based on the first sample image and the third sample image, a first loss value is obtained;

[0037] Image recognition is performed on the first sample image and the third sample image to obtain a first recognition result of the first sample image and a second recognition result of the second sample image. The first recognition result and the second recognition result are used to indicate whether the image is a generated image.

[0038] Based on the first identification result and the second identification result, a second loss value is obtained;

[0039] Based on the first loss value and the second loss value, obtain the target loss value.

[0040] In some embodiments, the first sample image, the second sample image, and the third sample image are face images;

[0041] The acquisition module is used for:

[0042] Obtain sample face images;

[0043] The sample face images are subjected to face detection and face registration processing;

[0044] Based on the processing results, the region where the face is located in the sample face image is cropped to obtain a first sample image, which includes the region where the face is located.

[0045] In some embodiments, the acquisition module is further configured to acquire a first image;

[0046] The super-resolution processing module is further configured to perform super-resolution image processing on the first image based on the image processing model to obtain a second image, wherein the resolution of the second image is greater than the resolution of the first image.

[0047] In some embodiments, the acquisition module is used to:

[0048] Get the video;

[0049] Face detection and face registration are performed on the face images in the video, where the face images are image frames in the video;

[0050] Based on the processing results, the region containing the face in the face image is cropped to obtain a first image, which includes the region containing the face.

[0051] In some embodiments, the apparatus further includes:

[0052] The fusion module is used to fuse the face image and the second image to obtain a third image, wherein the area where the face is located in the third image is the second image.

[0053] On one hand, an electronic device is provided, comprising one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement various optional implementations of the above-described image processing method.

[0054] On the one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored therein, the at least one computer program being loaded and executed by a processor to implement various optional implementations of the above-described image processing method.

[0055] In one aspect, a computer program product or computer program is provided, the computer program product or computer program comprising one or more lines of program code stored in a computer-readable storage medium. One or more processors of an electronic device are capable of reading the one or more lines of program code from the computer-readable storage medium, and the one or more processors execute the one or more lines of program code, enabling the electronic device to perform the image processing method of any of the above possible embodiments.

[0056] This application embodiment can randomly select a processing method to degrade a high-resolution first sample image to obtain a low-resolution second sample image. This eliminates the need for manual processing, significantly reducing labor costs and improving the efficiency of acquiring low-resolution images. Furthermore, during training, a different processing method can be selected for degradation processing each time. This ensures that the trained target image processing model can perform good image processing on images obtained through various methods, adapting to a wider range of image processing scenarios. Therefore, this target image processing model has good applicability. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a schematic diagram of the implementation environment of an image processing method provided in an embodiment of this application;

[0059] Figure 2 This is a flowchart of an image processing method provided in an embodiment of this application;

[0060] Figure 3 This is a flowchart of an image processing method provided in an embodiment of this application;

[0061] Figure 4 This is a schematic diagram of the structure of an image processing model provided in an embodiment of this application;

[0062] Figure 5 This is a schematic diagram of the structure of a residual dense module provided in an embodiment of this application;

[0063] Figure 6 This is a flowchart of an image processing method provided in an embodiment of this application;

[0064] Figure 7This is a flowchart of an image processing method provided in an embodiment of this application;

[0065] Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0066] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0067] Figure 10 This is a structural block diagram of a terminal provided in an embodiment of this application;

[0068] Figure 11 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0070] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0071] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple data packets means two or more data packets.

[0072] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing the particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0073] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0074] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] It should also be understood that determining B based on A does not mean determining B solely based on A; it is also possible to determine B based on A and / or other information.

[0076] It should also be understood that the term “comprising” (also referred to as “inCludes”, “inCluding”, “Comprises”, and / or “Comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0077] It should also be understood that the term "if" can be interpreted as meaning "when" or "upon" or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if determination..." or "if detection [the stated condition or event]" can be interpreted as meaning "when determination..." or "in response to determination..." or "when detection [the stated condition or event]" or "in response to detection [the stated condition or event]."

[0078] The image processing method provided in this application relates to the field of artificial intelligence technology. The following describes the related artificial intelligence technologies.

[0079] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0080] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0081] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and further performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0082] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0083] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0084] The solutions provided in this application involve image processing, video processing, and machine learning technologies in computer vision technology of artificial intelligence, and are specifically illustrated through the following embodiments.

[0085] The methods provided in this application relate to cloud technologies such as cloud computing, cloud social networking, cloud gaming, and artificial intelligence cloud services, which may involve image processing or video processing. A brief description of cloud technologies follows.

[0086] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0087] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0088] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed in the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0089] Based on logical function, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS (Infrastructure as a Service) layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be deployed directly on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally speaking, SaaS and PaaS are upper layers compared to IaaS.

[0090] The implementation environment of this application is described below.

[0091] Figure 1 This is a schematic diagram of an implementation environment for an image processing method provided in this application embodiment. The implementation environment includes a terminal 101, or it includes a terminal 101 and an image processing platform 102. The terminal 101 is connected to the image processing platform 102 via a wireless network or a wired network.

[0092] Terminal 101 can be at least one of the following: smartphone, game console, desktop computer, tablet computer, e-book reader, MP3 (Moving Picture Experts Group Audio Layer III) player or MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop computer, smart robot, or self-service payment device. Terminal 101 has an application that supports image processing installed and running; for example, the application can be a system application, instant messaging application, news push application, image processing application, video application, or social application.

[0093] For example, the terminal 101 can have image acquisition and image processing functions, and can process the acquired images and perform corresponding functions based on the processing results. The terminal 101 can complete this task independently, or it can be provided with data services by the image processing platform 102. This application embodiment does not limit this.

[0094] Image processing platform 102 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Image processing platform 102 provides background services for image processing applications. Optionally, image processing platform 102 undertakes the primary processing task, and terminal 101 undertakes secondary processing tasks; or, image processing platform 102 undertakes secondary processing tasks, and terminal 101 undertakes primary processing tasks; or, image processing platform 102 or terminal 101 can each independently undertake processing tasks. Alternatively, image processing platform 102 and terminal 101 can collaborate using a distributed computing architecture.

[0095] Optionally, the image processing platform 102 includes at least one server 1021 and a database 1022. The database 1022 is used to store data. In this embodiment, the database 1022 can store sample images, sample images or image processing models to provide data services to at least one server 1021.

[0096] Servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminals can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these.

[0097] Those skilled in the art will understand that the number of terminals 101 and servers 1021 can be more or less. For example, there may be only one terminal 101 or server 1021, or there may be dozens or hundreds of terminals 101 or servers 1021, or even more. The embodiments of this application do not limit the number or type of terminals or servers.

[0098] The application scenarios of this application are described below.

[0099] The image processing method provided in this application can be applied to various image processing scenarios. In some embodiments, the image processing scenario can be a scenario where a video application is used to play video. When a low-resolution video is obtained, the image processing model trained by the image processing method provided in this application can be used to process all or part of the image frames in the video to obtain a high-resolution video.

[0100] In other embodiments, the image processing scenario can be a scenario where an image processing application is used to capture images. After the images are captured, they can be processed using the image processing model described above to obtain higher resolution video.

[0101] The above provides two possible image processing scenarios. The image processing method provided in this application embodiment can also be applied to other image processing scenarios, which will not be listed here.

[0102] Figure 2 This is a flowchart of an image processing method provided in an embodiment of this application. The method is applied in an electronic device, which may be a terminal or a server. See also... Figure 2 The method includes the following steps.

[0103] 201. The electronic device acquires the first sample image.

[0104] The first sample image has a relatively high resolution and is a high-quality image. In this embodiment, the electronic device can acquire the high-resolution first sample image, and then perform degradation processing on it to obtain a low-resolution second sample image. This allows for rapid and efficient acquisition of sample images, reducing labor costs and improving the efficiency of sample image acquisition. Degradation processing refers to the process of converting a high-resolution image into a low-resolution image.

[0105] 202. An electronic device randomly selects one of at least two processing methods to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image.

[0106] These at least two processing methods are used to process high-resolution sample images into low-resolution sample images. By randomly selecting one processing method for degradation processing, the image processing model can be adapted to the image obtained by that degradation method.

[0107] 203. The electronic device performs super-resolution image processing on the second sample image based on the image processing model to obtain a third sample image, the resolution of which is greater than that of the second sample image.

[0108] The third sample image is obtained based on super-resolution image processing; therefore, the resolution of the third sample image is greater than that of the second sample image. This process is a high-resolution image reconstruction process. For the degraded second sample image, a high-resolution image, which is the third sample image, is reconstructed through super-resolution image processing.

[0109] 204. The electronic device obtains the target loss value based on the first sample image and the third sample image.

[0110] After the electronic device acquires the degraded and reconstructed third sample image, it can compare it with the original first sample image to determine the performance of the image processing model. The target loss value can be used to indicate the performance of the image processing model.

[0111] 205. The electronic device updates the model parameters of the image processing model based on the target loss value.

[0112] Electronic devices can update the model parameters of the current image processing model based on its performance to optimize the model's performance.

[0113] 206. The electronic device continues to execute the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value, until the target conditions are met and the process stops, thus obtaining the target image processing model.

[0114] Steps 202 to 204 above constitute an iterative process. During this iterative process, the target loss value can be obtained to measure the performance of the image processing model, thereby updating the model parameters and continuing the next iterative process. After multiple iterations, the image processing model can accurately process low-resolution images.

[0115] This application embodiment can randomly select a processing method to degrade a high-resolution first sample image to obtain a low-resolution second sample image. This eliminates the need for manual processing, significantly reducing labor costs and improving the efficiency of acquiring low-resolution images. Furthermore, during training, a different processing method can be selected for degradation processing each time. This ensures that the trained target image processing model can perform good image processing on images obtained through various methods, adapting to a wider range of image processing scenarios. Therefore, this target image processing model has good applicability.

[0116] Figure 3 This is a flowchart of an image processing method provided in an embodiment of this application. See also... Figure 3 The method includes the following steps.

[0117] 301. Electronic devices acquire sample face images.

[0118] In this embodiment, the electronic device can acquire sample face images and process them to obtain a first sample image that can be used to train an image processing model. After preliminary processing, more standardized or uniform sample images can be obtained, thereby improving the performance of the image processing model or reducing the training time and improving training efficiency.

[0119] The sample face image can be stored in different locations, and correspondingly, the electronic device can acquire the sample face image in different ways.

[0120] In some embodiments, the sample face image can be stored in an image database, and the electronic device can extract the sample face image from the image database accordingly. For example, the image database can be a face image library.

[0121] In other embodiments, the sample face image may also be stored in the electronic device, and the electronic device may extract the sample face image from local storage.

[0122] 302. The electronic device performs face detection and face registration processing on the sample face image.

[0123] Electronic devices can perform face detection and face registration on sample face images, which can determine the region where the face is located in the sample face image, as well as the direction or posture of the face. Based on this, the sample face image can be cropped to obtain a more standard and normalized sample image.

[0124] In some embodiments, the electronic device performs face detection on the sample face image to obtain a face detection result, and then performs face registration based on the face detection result. This face detection and face registration process is used to detect key points of the face and obtain a face detection box to roughly identify the face outline, thus revealing the region where the face is located in the sample face image.

[0125] 303. Based on the processing results, the electronic device crops the area where the face is located in the sample face image to obtain a first sample image, which includes the area where the face is located.

[0126] The processing result is used to indicate the region where the face is located in the sample face image. Based on the processing result, the electronic device can crop out the region where the face is located and then perform targeted analysis on the face to generate a high-resolution face image. In this way, the trained image processing model can accurately restore the face details, increase the clarity of the face image, and improve the quality of the face image.

[0127] In some embodiments, the cropping process can be achieved by cropping the area where the face is located in the processing result by a certain proportion. Specifically, the electronic device crops the sample face image based on the area where the face is located indicated by the processing result and the target proportion to obtain a first sample image, wherein the first sample image includes the area where the face is located and the pixels near the face.

[0128] The target ratio can be set by relevant technical personnel according to their needs. For example, the target ratio can be 120% or 150%. This application embodiment does not limit this.

[0129] Steps 301 to 303 describe the process of acquiring the first sample image. This process is explained using only the first sample image as a face image. Correspondingly, the second and third sample images in the following steps are also face images. Steps 301 and 303 are optional. In some embodiments, the first, second, and third sample images may not be face images. Therefore, the face detection, registration, and cropping steps described above do not need to be performed, and the electronic device can directly acquire the first sample image. The process of acquiring the first sample image can be similar to the process of acquiring the sample face image in step 301. This application embodiment does not limit this aspect.

[0130] 304. An electronic device randomly selects one of at least two processing methods to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image.

[0131] During each iteration of model training, electronic devices can first degrade the high-resolution first sample image to obtain a low-resolution second sample image. In this way, the first sample image and the second sample image form a sample image pair, which can be used as sample images in subsequent image processing to train the image processing model.

[0132] In some embodiments, the at least two processing methods include at least two of image blurring, adding image noise, image filtering, and image compression. These are all feasible degradation processing methods. Therefore, step 304 above includes any one of the following cases.

[0133] In scenario one, the electronic device blurs the first sample image to obtain the second sample image. For example, this blurring process can use random Gaussian blur or other blurring methods.

[0134] In scenario two, the electronic device adds image noise to the first sample image to obtain the second sample image.

[0135] In scenario three, the electronic device performs image filtering on the first sample image to obtain the second sample image. For example, this image filtering can be achieved through median filtering, average filtering, or other filtering methods.

[0136] In scenario four, the electronic device compresses the first sample image to obtain the second sample image. For example, this image compression can use JPEG (Joint Photographic Experts Group) compression or other compression methods.

[0137] In some embodiments, the degradation processing can be implemented using a degradation model. Specifically, the electronic device can randomly select a degradation model from at least two degradation models and process the first sample image based on the degradation model and its corresponding degradation parameters. This degradation model can be trained using high-resolution images and low-quality images captured from real-world scenes. For example, the training process can be performed using an unsupervised Pix2Pix method, or it can be trained using high-resolution images and low-quality images obtained through certain degradation processing methods. This application does not limit the scope of this embodiment.

[0138] In some embodiments, during each iteration of the model training process, the degradation processing can use different degradation parameters to obtain diverse sample images, which can better expand the sample images and obtain more comprehensive sample images, thereby improving the randomness of the sample images and thus improving the performance of the trained model. Specifically, the electronic device can randomly select a degradation parameter from at least two degradation parameters corresponding to the degradation model, and then process the first sample image based on the degradation model and the randomly selected degradation parameter. For example, the degradation parameter for random Gaussian noise is the std parameter, and the value of the std parameter can be 1.0 to 3.0, which can be randomly selected within this range during the iteration process. Another example is the average filtering method, where the degradation parameter is the k parameter, and the value of the k parameter can be 4 to 7. Yet another example is the median filtering method, where the degradation parameter is the k parameter, and the value of the k parameter can be 3 to 9. And yet another example is the C compression method, where the degradation parameter is the rate parameter, and the value of the rate parameter can be 50% to 90%. For example, in the Gaussian noise mode, the degradation parameter is the std parameter, which can take values ​​from 0 to 0.05.

[0139] 305. The electronic device performs super-resolution image processing on the second sample image based on an image processing model to obtain a third sample image, the resolution of which is greater than that of the second sample image.

[0140] The super-resolution image processing can be implemented through residual dense modules. Specifically, the electronic device can input the second sample image into the image processing model, and the second sample image is subjected to super-resolution image processing by at least two residual dense modules in the image processing model to obtain at least two residual images. Based on the at least two residual images and the second sample image, a third sample image is obtained and output. The at least two residual images are obtained by super-resolution image processing based on a different number of residual dense modules.

[0141] In some embodiments, the number of residual dense modules is one or more, and these residual dense modules are connected in series. However, short connections can be used between the residual dense modules. Different levels of features can be extracted through different numbers of residual dense modules. The model parameters include the weights of at least two residual images. In this process, the electronic device fuses the at least two residual images with the second sample image respectively to obtain at least two candidate third sample images. Then, based on the weights of the at least two residual images, the at least two candidate third sample images can be weighted to obtain the third sample image, and the third sample image is output. For example, as shown... Figure 4 As shown, the convolutional network in the Entry module 401 converts the 3-channel input image into a 64-channel feature map. Inside the RDB (Residual Dense Block) 402, dense connections concatenate features from different layers, resulting in a wider network and improved generalization ability. A short residual connection is added between the input and output of the RDB to improve training stability. Each residual dense block outputs a residual image. The Exit module 403 converts the feature maps output from multiple RDB modules into 3-channel residual images. These residual images are then added to the input image to obtain the reconstructed image. Note that the Exit module has multiple outputs, which can be obtained through weighted averaging. The weights w are learnable variables, i.e., model parameters, and are automatically updated during network training.

[0142] For example, this weighted processing can be implemented using the following formula:

[0143]

[0144] Wherein, input is the second sample image, output is the third sample image, and w iLet M be the weight of each of the at least two residual images. The value of M is only one example.

[0145] For a residual dense module, it can include at least two convolutional layers cascaded using dense connections. Through multiple convolutional processes, high-dimensional information can be extracted, supplementing details in the second sample image to obtain a higher-resolution third sample image. For example, as... Figure 5 As shown, the residual dense module includes multiple convolutional layers 501 (referred to as Conv2d). After each convolutional layer performs convolution processing on the data, it can be rectified by a rectified linear unit (ReLU) 502 before being input into the next convolution. Finally, the residual dense module can concat 503 the results of the previous multiple convolutional layers and output them after performing a single convolution by a convolutional layer 504.

[0146] 306. The electronic device obtains the target loss value based on the first sample image and the third sample image.

[0147] In some embodiments, the target loss value may include two loss values, and this training process can be implemented using a Generative Adversarial Network (GAN). A GAN is an unsupervised learning method consisting of a generator network and a discriminator network. The discriminator network takes either a real sample or the output of the generator network as input, and its goal is to distinguish the generator network's output from the real samples as closely as possible. The generator network, on the other hand, aims to deceive the discriminator network as much as possible. The two networks compete against each other, continuously adjusting their parameters, ultimately generating highly realistic images.

[0148] Specifically, the electronic device can obtain a first loss value based on the first sample image and the third sample image, perform image recognition on the first sample image and the third sample image to obtain a first recognition result of the first sample image and a second recognition result of the second sample image, the first recognition result and the second recognition result being used to indicate whether the image is a generated image; obtain a second loss value based on the first recognition result and the second recognition result; and obtain a target loss value based on the first loss value and the second loss value.

[0149] In some embodiments, the electronic device can distinguish between the first sample image and the third sample image to determine whether the two images are generated images or real images. This discrimination process can be implemented by a discriminator or a discriminative network.

[0150] For example, such as Figure 6 As shown, the electronic device can acquire a first sample image 601, which has a high resolution and is relatively clear. The electronic device can then degrade this image 601 using a degradation model 602 to obtain a second sample image 603, which has a lower resolution and is relatively blurry. The electronic device can use the second sample image 603 as input to a restoration model 604, and restore the second sample image 603 using the restoration model 604 to obtain a high-resolution output 605. The first sample image 601 is the high-resolution image (GT). The output 605 of the restoration model 604 can be discriminated by a discriminator 606, and based on the discriminant, an adversarial loss (GAN loss) 607 is determined. Based on the output 605 of the restoration model 604 and the GT, a reconstruction loss 608 can also be determined. This first loss value can be called the reconstruction loss, denoted as L. LPIPS The second loss value can be called the adversarial loss (GAN loss), denoted as L. GAN This is used to ensure that the distribution of the generated results matches the distribution of the real input graph, thereby improving the realism of the results. The process of obtaining this target loss value can be achieved through formulas two, three, and four:

[0151] L = L GAN +L LPIPS Formula 2

[0152]

[0153] L LPIPS =|G(input)-GT|1, formula 4

[0154] Where L is the target loss value, L GAN For the second loss value, L LPIPS is the first loss value. G(input) is the output obtained after super-resolution image processing of the input, which is the third sample image. GT is the high-resolution image, which is the first sample image. E[] is the expectation. log() is the logarithmic function. D() is the result obtained by the discriminator in judging the image.

[0155] 307. The electronic device updates the model parameters of the image processing model based on the target loss value.

[0156] This model parameter update process is used to optimize the performance of the image processing model, ultimately enabling the model to accurately perform super-resolution image processing. This update process can be implemented using gradient descent, or other methods; there are no limitations on the specific method used.

[0157] 308. The electronic device continues to execute the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stops to obtain the target image processing model.

[0158] The steps repeated in step 308 are the same as those in steps 304 to 307 above, and will not be elaborated further here.

[0159] The target condition can be set by relevant technical personnel according to their needs, such as reaching a target number of iterations or convergence of the target loss value. This application does not limit this.

[0160] For example, in a specific example, such as Figure 7 As shown, in the above image processing process, for the network face data 701, a detection, registration and cropping step 702 can be performed, followed by data filtering 703 to obtain a high-definition portrait 704. Then, the high-definition portrait 704 can be processed by a degradation model 705 to obtain low-quality data 706. Finally, the high-definition portrait 704 and the low-quality data 706 are combined to train a high-definition portrait restoration model 707 (that is, an image processing model).

[0161] 309. The electronic device acquires the first image.

[0162] Electronic devices can acquire a first image in various ways, and this acquisition process can be implemented differently in different image processing scenarios. Two possible methods are provided below.

[0163] In Method 1, the electronic device can acquire a face image, perform face detection and face registration processing on the face image, and based on the processing results, crop the region where the face is located in the face image to obtain a first image, which includes the region where the face is located. In Method 1, the image processing scenario involves processing the acquired face image (e.g., a captured face image) to improve the image resolution.

[0164] In Method Two, the electronic device acquires a video and performs face detection and registration processing on the face images in the video. The face images are image frames from the video. Based on the processing results, the region containing the face in the face image is cropped to obtain a first image, which includes the region containing the face. In Method Two, the image processing scenario involves processing the acquired video to improve its resolution. The electronic device can acquire only a portion of the video frames as face images, or it can acquire all video frames as face images, or it can extract frames from the video, perform face detection on the extracted video frames, and when a video frame contains a face, it is used as a face image for further processing. This application embodiment does not limit this approach.

[0165] 310. The electronic device performs super-resolution image processing on the first image based on the image processing model to obtain a second image, the resolution of which is greater than that of the first image.

[0166] Step 310 is the same as step 305 above, and will not be elaborated on here.

[0167] In some embodiments, after processing the first image to obtain a higher-resolution second image, the electronic device can further fuse the face image and the second image to obtain a third image, in which the area containing the face is the same as that in the second image. For example, the electronic device can replace the corresponding area of ​​the first image in the face image with the second image to obtain the third image. In this way, the area containing the face in the face image is replaced by the second image to obtain the third image. The third image supplements the facial details and has a higher resolution; therefore, the third image is a high-definition face image after quality optimization of the face image.

[0168] This application embodiment can randomly select a processing method to degrade a high-resolution first sample image to obtain a low-resolution second sample image. This eliminates the need for manual processing, significantly reducing labor costs and improving the efficiency of acquiring low-resolution images. Furthermore, during training, a different processing method can be selected for degradation processing each time. This ensures that the trained target image processing model can perform good image processing on images obtained through various methods, adapting to a wider range of image processing scenarios. Therefore, this target image processing model has good applicability.

[0169] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0170] Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. See also... Figure 8 The device includes:

[0171] The acquisition module 801 is used to acquire the first sample image;

[0172] The degradation processing module 802 is used to randomly select one processing method from at least two processing methods to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is smaller than the resolution of the first sample image.

[0173] The super-resolution processing module 803 is used to perform super-resolution image processing on the second sample image based on the image processing model to obtain a third sample image, the resolution of which is greater than that of the second sample image.

[0174] The loss value acquisition module 804 is used to acquire the target loss value based on the first sample image and the third sample image;

[0175] The update module 805 is used to update the model parameters of the image processing model based on the target loss value.

[0176] The device continues to execute the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stops to obtain the target image processing model.

[0177] In some embodiments, the at least two processing methods include at least two of image blurring, adding image noise, image filtering, and image compression;

[0178] The degradation processing module 802 is used for any of the following:

[0179] The first sample image is blurred to obtain the second sample image;

[0180] Add image noise to the first sample image to obtain the second sample image;

[0181] The first sample image is filtered to obtain the second sample image;

[0182] The first sample image is compressed to obtain the second sample image.

[0183] In some embodiments, the degradation processing module 802 is used to randomly select a degradation model from at least two degradation models, and process the first sample image based on the degradation model and the degradation parameters corresponding to the degradation model.

[0184] In some embodiments, the degradation processing module 802 is used for:

[0185] From at least two degradation parameters corresponding to the degradation model, randomly select one degradation parameter;

[0186] The first sample image is processed based on the degradation model and the randomly selected degradation parameters.

[0187] In some embodiments, the super-resolution processing module 803 is used to input the second sample image into the image processing model, and to perform super-resolution image processing on the second sample image by at least two residual dense modules in the image processing model to obtain at least two residual images. Based on the at least two residual images and the second sample image, a third sample image is obtained and output; wherein the at least two residual images are obtained by performing super-resolution image processing based on a different number of residual dense modules.

[0188] In some embodiments, the model parameters include the weights of at least two residual images;

[0189] The super-resolution processing module 803 is used for:

[0190] The at least two residual images are fused with the second sample image respectively to obtain at least two candidate third sample images;

[0191] Based on the weights of the at least two residual images, the at least two candidate third sample images are weighted to obtain the third sample image, and the third sample image is output.

[0192] In some embodiments, the residual dense module includes at least two convolutional layers connected in series with dense links.

[0193] In some embodiments, the loss value acquisition module 804 is used for:

[0194] Based on the first sample image and the third sample image, obtain the first loss value;

[0195] Image recognition is performed on the first sample image and the third sample image to obtain a first recognition result of the first sample image and a second recognition result of the second sample image. The first recognition result and the second recognition result are used to indicate whether the image is a generated image.

[0196] Based on the first identification result and the second identification result, a second loss value is obtained;

[0197] Based on the first loss value and the second loss value, the target loss value is obtained.

[0198] In some embodiments, the first sample image, the second sample image, and the third sample image are face images;

[0199] The acquisition module 801 is used for:

[0200] Obtain sample face images;

[0201] Face detection and face registration were performed on the sample face image;

[0202] Based on the processing results, the region where the face is located in the sample face image is cropped to obtain a first sample image, which includes the region where the face is located.

[0203] In some embodiments, the acquisition module 801 is further configured to acquire a first image;

[0204] The super-resolution processing module 803 is also used to perform super-resolution image processing on the first image based on the image processing model to obtain a second image, the resolution of which is greater than that of the first image.

[0205] In some embodiments, the acquisition module 801 is used to:

[0206] Get the video;

[0207] Face detection and face registration are performed on the face images in the video, which are image frames in the video;

[0208] Based on the processing results, the area containing the face in the face image is cropped to obtain a first image, which includes the area containing the face.

[0209] In some embodiments, the device further includes:

[0210] The fusion module is used to fuse the face image and the second image to obtain a third image, in which the face area is the same as that in the second image.

[0211] It should be noted that the image processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the image processing apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and the image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0212] Figure 9This is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of this application. The electronic device 900 can vary considerably depending on its configuration or performance. It can include one or more Central Processing Units (CPUs) 901 and one or more memories 902. The memory 902 stores at least one computer program, which is loaded and executed by the processor 901 to implement the image processing methods provided in the various method embodiments described above. The electronic device can also include other components for implementing device functions. For example, the electronic device can also have wired or wireless network interfaces and input / output interfaces for input and output. Further details are not elaborated upon in this embodiment.

[0213] The electronic device in the above method embodiments can be implemented as a terminal. For example, Figure 10 This is a structural block diagram of a terminal provided in an embodiment of this application. The terminal 1000 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 (Moving Picture Experts Group Audio Layer III) player, MP4 (Moving Picture Experts Group Audio Layer IV) player, laptop computer, or desktop computer. The terminal 1000 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0214] Typically, terminal 1000 includes a processor 1001 and a memory 1002.

[0215] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1001 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0216] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1002 are used to store at least one instruction, which is executed by the processor 1001 to implement the image processing method provided in the method embodiments of this application.

[0217] In some embodiments, the terminal 1000 may also optionally include a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1004, a display screen 1005, a camera assembly 1006, an audio circuit 1007, a positioning assembly 1008, and a power supply 1009.

[0218] Peripheral device interface 1003 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1001 and memory 1002. In some embodiments, processor 1001, memory 1002 and peripheral device interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1001, memory 1002 and peripheral device interface 1003 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0219] The radio frequency (RF) circuit 1004 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1004 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1004 can communicate with other terminals via at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1004 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0220] Display screen 1005 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1005 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1001 for processing. In this case, display screen 1005 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1005, disposed on the front panel of terminal 1000; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal 1000 or in a folded design; in still other embodiments, display screen 1005 may be a flexible display screen, disposed on a curved or folded surface of terminal 1000. Furthermore, display screen 1005 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1005 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0221] The camera assembly 1006 is used to acquire images or videos. Optionally, the camera assembly 1006 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0222] The audio circuit 1007 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1001 for processing, or input to the radio frequency circuit 1004 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 1000. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1001 or the radio frequency circuit 1004 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1007 may also include a headphone jack.

[0223] The positioning component 1008 is used to determine the current geographical location of the terminal 1000 in order to enable navigation or LBS (Location Based Service). The positioning component 1008 can be a positioning component based on the US GPS (Global Positioning System), China's BeiDou system, or Russia's Galileo system.

[0224] Power supply 1009 is used to power the various components in terminal 1000. Power supply 1009 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1009 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0225] In some embodiments, the terminal 1000 further includes one or more sensors 1010. The one or more sensors 1010 include, but are not limited to: an accelerometer 1011, a gyroscope 1012, a pressure sensor 1013, a fingerprint sensor 1014, an optical sensor 1015, and a proximity sensor 1016.

[0226] Accelerometer 1011 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 1000. For example, accelerometer 1011 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1001 can control display screen 1005 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1011. Accelerometer 1011 can also be used for games or for acquiring user motion data.

[0227] The gyroscope sensor 1012 can detect the orientation and rotation angle of the terminal 1000. The gyroscope sensor 1012, in conjunction with the accelerometer sensor 1011, can collect 3D motion data from the user on the terminal 1000. Based on the data collected by the gyroscope sensor 1012, the processor 1001 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0228] The pressure sensor 1013 can be disposed on the side bezel of the terminal 1000 and / or on the lower layer of the display screen 1005. When the pressure sensor 1013 is disposed on the side bezel of the terminal 1000, it can detect the user's grip signal on the terminal 1000, and the processor 1001 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1013. When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1005. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0229] The fingerprint sensor 1014 is used to collect a user's fingerprint. The processor 1001 identifies the user based on the fingerprint collected by the fingerprint sensor 1014, or vice versa. When the user's identity is identified as trusted, the processor 1001 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1014 can be located on the front, back, or side of the terminal 1000. When the terminal 1000 has physical buttons or a manufacturer's logo, the fingerprint sensor 1014 can be integrated with the physical buttons or manufacturer's logo.

[0230] An optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 based on the ambient light intensity collected by the optical sensor 1015. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 can also dynamically adjust the shooting parameters of the camera assembly 1006 based on the ambient light intensity collected by the optical sensor 1015.

[0231] The proximity sensor 1016, also known as a distance sensor, is typically mounted on the front panel of the terminal 1000. The proximity sensor 1016 is used to detect the distance between the user and the front of the terminal 1000. In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the terminal 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from a screen-on state to a screen-off state; when the proximity sensor 1016 detects that the distance between the user and the front of the terminal 1000 is gradually increasing, the processor 1001 controls the display screen 1005 to switch from a screen-off state to a screen-on state.

[0232] Those skilled in the art will understand that Figure 10 The structure shown does not constitute a limitation on terminal 1000 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0233] The electronic device in the above method embodiments can be implemented as a server. For example, Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly depending on its configuration or performance. It can include one or more Central Processing Units (CPUs) 1101 and one or more memories 1102. The memories 1102 store at least one computer program, which is loaded and executed by the processor 1101 to implement the image processing methods provided in the various method embodiments described above. Of course, the server can also have wired or wireless network interfaces and input / output interfaces for input and output. The server can also include other components for implementing device functions, which will not be elaborated here.

[0234] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including at least one computer program, which is executable by a processor to perform the image processing method described above. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0235] In an exemplary embodiment, a computer program product or computer program is also provided, comprising one or more computer programs stored in a computer-readable storage medium. One or more processors of an electronic device are capable of reading the one or more computer programs from the computer-readable storage medium, and the one or more processors execute the one or more computer programs, enabling the electronic device to perform the image processing method described above.

[0236] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0237] It should be understood that determining B based on A does not mean determining B solely based on A; it also means determining B based on A and / or other information.

[0238] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0239] The above description is only an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the first sample image; From at least two processing methods, one processing method is randomly selected to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image; The second sample image is input into the image processing model, and at least two residual dense modules in the image processing model perform super-resolution image processing on the second sample image to obtain at least two residual images. Based on the at least two residual images and the second sample image, a third sample image is obtained and output. The at least two residual images are obtained by super-resolution image processing based on a different number of residual dense modules, and the resolution of the third sample image is greater than the resolution of the second sample image. Based on the first sample image and the third sample image, the target loss value is obtained; Based on the target loss value, the model parameters of the image processing model are updated; Continue executing the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stop to obtain the target image processing model.

2. The method according to claim 1, characterized in that, The at least two processing methods include at least two of image blurring, adding image noise, image filtering, and image compression. The step of randomly selecting one processing method from at least two processing methods to process the first sample image to obtain the second sample image includes any one of the following: The first sample image is blurred to obtain the second sample image; Add image noise to the first sample image to obtain the second sample image; The first sample image is filtered to obtain the second sample image; The first sample image is compressed to obtain the second sample image.

3. The method according to claim 1, characterized in that, The step of randomly selecting one processing method from at least two processing methods to process the first sample image includes: From at least two degradation models, a degradation model is randomly selected, and the first sample image is processed based on the degradation model and the degradation parameters corresponding to the degradation model.

4. The method according to claim 3, characterized in that, The processing of the first sample image based on the degradation model and the degradation parameters corresponding to the degradation model includes: From at least two degradation parameters corresponding to the degradation model, randomly select one degradation parameter; The first sample image is processed based on the degradation model and the randomly selected degradation parameters.

5. The method according to claim 1, characterized in that, The model parameters include the weights of at least two residual images; The step of acquiring and outputting a third sample image based on the at least two residual images and the second sample image includes: The at least two residual images are fused with the second sample image to obtain at least two candidate third sample images; Based on the weights of the at least two residual images, the at least two candidate third sample images are weighted to obtain the third sample image, and the third sample image is output.

6. The method according to claim 1, characterized in that, The residual dense module includes at least two convolutional layers connected in series with dense links.

7. The method according to claim 1, characterized in that, The step of obtaining the target loss value based on the first sample image and the third sample image includes: Based on the first sample image and the third sample image, a first loss value is obtained; Image recognition is performed on the first sample image and the third sample image to obtain a first recognition result of the first sample image and a second recognition result of the second sample image. The first recognition result and the second recognition result are used to indicate whether the image is a generated image. Based on the first identification result and the second identification result, a second loss value is obtained; Based on the first loss value and the second loss value, obtain the target loss value.

8. The method according to claim 1, characterized in that, The first sample image, the second sample image, and the third sample image are face images; The acquisition of the first sample image includes: Obtain sample face images; The sample face images are subjected to face detection and face registration processing; Based on the processing results, the region where the face is located in the sample face image is cropped to obtain a first sample image, which includes the region where the face is located.

9. The method according to claim 1, characterized in that, The method further includes: Get the first image; The first image is subjected to super-resolution image processing based on the image processing model to obtain a second image, the resolution of which is greater than that of the first image.

10. The method according to claim 9, characterized in that, The acquisition of the first image includes: Get the video; Face detection and face registration are performed on the face images in the video, where the face images are image frames in the video; Based on the processing results, the region containing the face in the face image is cropped to obtain a first image, which includes the region containing the face.

11. The method according to claim 9 or 10, characterized in that, The method further includes: The face image and the second image are fused to obtain a third image, wherein the area where the face is located in the third image is the second image.

12. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the first sample image; The degradation processing module is used to randomly select one processing method from at least two processing methods to process the first sample image to obtain a second sample image, wherein the resolution of the second sample image is less than the resolution of the first sample image; The super-resolution processing module is used to input the second sample image into the image processing model, where at least two residual dense modules in the image processing model perform super-resolution image processing on the second sample image to obtain at least two residual images. Based on the at least two residual images and the second sample image, a third sample image is obtained and output. The at least two residual images are obtained by performing super-resolution image processing on a different number of residual dense modules, and the resolution of the third sample image is greater than the resolution of the second sample image. The loss value acquisition module is used to acquire a target loss value based on the first sample image and the third sample image; An update module is used to update the model parameters of the image processing model based on the target loss value; The device continues to execute the steps of randomly selecting a processing method to process the first sample image, performing super-resolution image processing based on the image processing model, and obtaining the target loss value until the target conditions are met, and then stops to obtain the target image processing model.

13. The apparatus according to claim 12, characterized in that, The at least two processing methods include at least two of image blurring, adding image noise, image filtering, and image compression. The degradation processing module is used for any of the following: The first sample image is blurred to obtain the second sample image; Add image noise to the first sample image to obtain the second sample image; The first sample image is filtered to obtain the second sample image; The first sample image is compressed to obtain the second sample image.

14. The apparatus according to claim 12, characterized in that, The degradation processing module is used for: From at least two degradation models, a degradation model is randomly selected, and the first sample image is processed based on the degradation model and the degradation parameters corresponding to the degradation model.

15. The apparatus according to claim 14, characterized in that, The degradation processing module is used for: From at least two degradation parameters corresponding to the degradation model, randomly select one degradation parameter; The first sample image is processed based on the degradation model and the randomly selected degradation parameters.

16. The apparatus according to claim 12, characterized in that, The model parameters include the weights of at least two residual images; The super-resolution processing module is used for: The at least two residual images are fused with the second sample image to obtain at least two candidate third sample images; Based on the weights of the at least two residual images, the at least two candidate third sample images are weighted to obtain the third sample image, and the third sample image is output.

17. The apparatus according to claim 12, characterized in that, The residual dense module includes at least two convolutional layers connected in series with dense links.

18. The apparatus according to claim 12, characterized in that, The loss value acquisition module is used for: Based on the first sample image and the third sample image, a first loss value is obtained; Image recognition is performed on the first sample image and the third sample image to obtain a first recognition result of the first sample image and a second recognition result of the second sample image. The first recognition result and the second recognition result are used to indicate whether the image is a generated image. Based on the first identification result and the second identification result, a second loss value is obtained; Based on the first loss value and the second loss value, obtain the target loss value.

19. The apparatus according to claim 12, characterized in that, The first sample image, the second sample image, and the third sample image are face images; The acquisition module is used for: Obtain sample face images; The sample face images are subjected to face detection and face registration processing; Based on the processing results, the region where the face is located in the sample face image is cropped to obtain a first sample image, which includes the region where the face is located.

20. The apparatus according to claim 12, characterized in that, The acquisition module is also used to acquire the first image; The super-resolution processing module is also used to perform super-resolution image processing on the first image based on the image processing model to obtain a second image, wherein the resolution of the second image is greater than the resolution of the first image.

21. The apparatus according to claim 20, characterized in that, The acquisition module is used for: Get the video; Face detection and face registration are performed on the face images in the video, where the face images are image frames in the video; Based on the processing results, the region containing the face in the face image is cropped to obtain a first image, which includes the region containing the face.

22. The apparatus according to claim 20 or 21, characterized in that, The device further includes: The fusion module is used to fuse the face image and the second image to obtain a third image, wherein the area where the face is located in the third image is the second image.

23. An electronic device, characterized in that, The electronic device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the image processing method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the image processing method as described in any one of claims 1 to 11.

25. A computer program product, characterized in that, The computer program product includes one or more computer programs stored in a computer-readable storage medium, one or more processors of an electronic device reading the one or more computer programs from the computer-readable storage medium, and the one or more processors executing the one or more computer programs to cause the electronic device to perform an image processing method as described in any one of claims 1 to 11.

Images