Training of an image processing model, image processing method, apparatus, device and medium

Abstract

The application discloses a kind of training of image processing model, image processing method, device, equipment and medium, belong to artificial intelligence technical field.The training method of the image processing model includes: obtaining sample image with illumination effect and the label image corresponding to sample image without illumination effect, sample image is obtained by image acquisition to the virtual model of reference object illuminated by virtual light source;Sample image and label image are used to train initial image processing model, and target image processing model is obtained.This kind of mode, the acquisition cost of sample image is lower and can be more real simulation to the image to be processed with illumination effect, the effect of training image processing model is better, it is favorable to improve the reliability of using trained image processing model to the image to be processed with illumination effect and carry out de-illumination processing.The application embodiment can be applied to cloud technology, artificial intelligence, intelligent transportation, various scenes such as auxiliary driving.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence, and in particular to training of an image processing model, an image processing method and device, equipment and a medium. BACKGROUND

[0002] With the development of artificial intelligence technology, there are more and more application scenarios of using an image processing model to process an image to obtain another image. For example, an image processing model is used to perform de-lighting processing on a to-be-processed image with lighting effects to obtain an image without lighting effects, wherein the to-be-processed image is obtained by image acquisition on a reference object irradiated by a real light source.

[0003] Before using the image processing model to perform de-lighting processing on the to-be-processed image with lighting effects, the image processing model needs to be trained first. How to train an image processing model that can perform relatively reliable de-lighting processing on a to-be-processed image with lighting effects at a lower cost to improve the quality of image processing is a problem that needs to be solved. SUMMARY

[0004] Embodiments of the present application provide a training method of an image processing model, an image processing method and device, equipment and a medium, which can be used to train an image processing model that can perform relatively reliable de-lighting processing on a to-be-processed image with lighting effects at a lower cost to improve the quality of image processing. The technical solution is as follows:

[0005] In one aspect, the present application provides a training method of an image processing model, which comprises:

[0006] Obtaining a sample image with lighting effects and a label image without lighting effects corresponding to the sample image, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, and the virtual light source being obtained by physical simulation on a real light source;

[0007] Training an initial image processing model using the sample image and the label image to obtain a target image processing model, the target image processing model being used to perform de-lighting processing on a to-be-processed image with lighting effects, the to-be-processed image being obtained by image acquisition on the reference object irradiated by a real light source.

[0008] An image processing method is also provided, which comprises:

[0009] obtain a to-be-processed image with lighting effects and a target image processing model, the target image processing model being trained by using a sample image with lighting effects and a label image without lighting effects corresponding to the sample image, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source being obtained by physical simulation on a real light source, and the to-be-processed image being obtained by image acquisition on the reference object irradiated by the real light source;

[0010] call the target image processing model to perform de-lighting processing on the to-be-processed image, and obtain a target image.

[0011] In another aspect, a training device of an image processing model is provided, and the device comprises:

[0012] an obtaining unit, configured to obtain a sample image with lighting effects and a label image without lighting effects corresponding to the sample image, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source being obtained by physical simulation on a real light source;

[0013] a training unit, configured to train an initial image processing model by using the sample image and the label image, and obtain a target image processing model, the target image processing model being used to perform de-lighting processing on a to-be-processed image with lighting effects, the to-be-processed image being obtained by image acquisition on the reference object irradiated by the real light source.

[0014] In a possible implementation, the obtaining unit is configured to load the virtual model of the reference object into a virtual light source environment, the virtual light source environment being provided with a virtual light source; perform image acquisition on the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment, and obtain the sample image; and perform image acquisition on the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment, and obtain the label image.

[0015] In a possible implementation, the training unit is configured to obtain a first texture image corresponding to the sample image and a second texture image corresponding to the label image; input the first texture image into the initial image processing model to perform de-lighting processing, and obtain a third texture image output by the initial image processing model; obtain a first loss function based on the third texture image and the second texture image; and train the initial image processing model by using the first loss function.

[0016] In a possible implementation, the initial image processing model comprises a generator and a discriminator, the third texture image is obtained by inputting the first texture image into the generator in the initial image processing model for de-illumination processing; the training unit is configured to input the second texture image into the discriminator in the initial image processing model for discrimination to obtain a first discrimination result output by the discriminator in the initial image processing model, the first discrimination result being used to indicate a probability that the second texture image is an image output by the generator; input the third texture image into the discriminator in the initial image processing model for discrimination to obtain a second discrimination result output by the discriminator in the initial image processing model, the second discrimination result being used to indicate a probability that the third texture image is an image output by the generator; obtain a first sub-loss function based on the first discrimination result and the second discrimination result; and obtain the first loss function based on the first sub-loss function.

[0017] In a possible implementation, the training unit is configured to determine a first sub-image in the second texture image that satisfies a reliable condition and a second sub-image in the third texture image that satisfies the reliable condition; obtain a second sub-loss function based on the first sub-image and the second sub-image; and obtain the first loss function based on the first sub-loss function and the second sub-loss function.

[0018] In a possible implementation, the training unit is configured to determine a texture point corresponding to a virtual model point in a texture space and a mapping point corresponding to the virtual model point in the sample image, the virtual model point being a point of a virtual model of the reference object; determine a pixel value of the texture point based on a pixel value of the mapping point in the sample image; and obtain the first texture image based on the texture point and the pixel value of the texture point.

[0019] In a possible implementation, the training unit is configured to input the sample image into the initial image processing model for de-illumination processing to obtain a predicted image output by the initial image processing model; obtain a second loss function based on the predicted image and the label image; and train the initial image processing model by using the second loss function.

[0020] An image processing apparatus is also provided, and the apparatus comprises:

[0021] The acquisition unit is configured to acquire a to-be-processed image with lighting effects and a target image processing model, the target image processing model is trained by using a sample image with lighting effects and a label image without lighting effects corresponding to the sample image, the sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source is obtained by physical simulation on a real light source, and the to-be-processed image is obtained by image acquisition on the reference object irradiated by the real light source.

[0022] The processing unit is configured to call the target image processing model to perform de-lighting processing on the to-be-processed image to obtain a target image.

[0023] In a possible implementation, the processing unit is configured to acquire a texture image corresponding to the to-be-processed image, input the texture image corresponding to the to-be-processed image into the target image processing model to perform de-lighting processing, and obtain the target image output by the target image processing model.

[0024] In a possible implementation, the processing unit is configured to input the to-be-processed image into the target image processing model to perform de-lighting processing, and obtain the target image output by the target image processing model.

[0025] In another aspect, a computer device is provided, which includes a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor, so that the computer device implements the image processing model training method or the image processing method described above.

[0026] In another aspect, a computer readable storage medium is also provided, which stores at least one computer program, which is loaded and executed by a processor, so that the computer implements the image processing model training method or the image processing method described above.

[0027] In another aspect, a computer program product is also provided, which includes a computer program or computer instructions, which are loaded and executed by a processor, so that the computer implements the image processing model training method or the image processing method described above.

[0028] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects:

[0029] The technical scheme provided in the embodiments of the present application is that the sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, and the sample image is obtained without irradiation of a real light source, and the cost is low. In addition, the virtual light source obtained by physical simulation of the real light source can simulate the real light source more realistically, and the virtual model of the reference object can simulate the reference object more realistically, so that the sample image with the lighting effect can simulate the to-be-processed image with the lighting effect more realistically, the training effect of the image processing model by using the sample image and the corresponding label image is good, and the reliability of the to-be-processed image with the lighting effect processed by using the trained image processing model is improved, and the image processing quality is high. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0031] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0032] Figure 2 is a flowchart of a training method of an image processing model provided by an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a sample image and a label image provided by an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of a sample image and a first texture image provided by an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of a first texture image and a third texture image provided by an embodiment of the present application;

[0036] Figure 6 is a schematic diagram of a mask image provided by an embodiment of the present application;

[0037] Figure 7 is a flowchart of an image processing method provided by an embodiment of the present application;

[0038] Figure 8 is a schematic diagram of a texture image provided by an embodiment of the present application;

[0039] Figure 9 is a schematic diagram of a training device of an image processing model provided by an embodiment of the present application;

[0040] Figure 10 is a schematic diagram of an image processing device provided by an embodiment of the present application;

[0041] Figure 11 is a structural schematic diagram of a server provided by an embodiment of the present application;

[0042] Figure 12 is a structural schematic diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0044] In exemplary embodiments, the training method of the image processing model and the image processing method provided by the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, intelligent transportation, and auxiliary driving.

[0045] Artificial intelligence (AI) is to use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0046] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation, etc. The training method of the image processing model and the image processing method provided by the embodiments of the present application involve computer vision technology and machine learning technology.

[0047] Computer Vision (CV) is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further perform image processing to make computer processing more suitable for human eye observation or image transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D (Three Dimensional) technology, virtual reality, augmented reality, map construction, autonomous driving, intelligent transportation, and other technologies. It also includes common biometric identification technologies.

[0048] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It is a specialized study of how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0049] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, intelligent medical care, intelligent customer service, Internet of Vehicles, autonomous driving, intelligent transportation, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0050] Figure 1 A schematic diagram of an implementation environment provided by an embodiment of the present application is shown. The implementation environment includes a terminal 11 and a server 12.

[0051] The training method of the image processing model provided in the embodiments of the present application can be executed by the terminal 11, can be executed by the server 12, and can also be executed by the terminal 11 and the server 12 together, and the embodiments of the present application do not limit this. For the case that the training method of the image processing model provided in the embodiments of the present application is executed by the terminal 11 and the server 12 together, the server 12 undertakes the main computing work, and the terminal 11 undertakes the secondary computing work; or the server 12 undertakes the secondary computing work, and the terminal 11 undertakes the main computing work; or the server 12 and the terminal 11 adopt a distributed computing architecture to perform collaborative computing.

[0052] The image processing method provided in the embodiments of the present application can be executed by the terminal 11, can be executed by the server 12, and can also be executed by the terminal 11 and the server 12 together, and the embodiments of the present application do not limit this. For the case that the image processing method provided in the embodiments of the present application is executed by the terminal 11 and the server 12 together, the server 12 undertakes the main computing work, and the terminal 11 undertakes the secondary computing work; or the server 12 undertakes the secondary computing work, and the terminal 11 undertakes the main computing work; or the server 12 and the terminal 11 adopt a distributed computing architecture to perform collaborative computing.

[0053] The training method of the image processing model and the image processing method provided in the embodiments of the present application can be executed by the same device or can be executed by different devices, and the embodiments of the present application do not limit this.

[0054] In a possible implementation manner, the terminal 11 can be any electronic product that can interact with a user through one or more manners such as a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, or a handwriting device, for example, a PC (Personal Computer), a mobile phone, a smart phone, a PDA (Personal Digital Assistant), a wearable device, a PPC (Pocket PC), a tablet computer, a smart car machine, a smart television, a smart sound box, a smart voice interaction device, a smart home appliance, a vehicle terminal, and the like. The server 12 can be a server or a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 and the server 12 establish a communication connection through a wired or wireless network.

[0055] Those skilled in the art shall understand that the terminal 11 and the server 12 described above are only examples, and other existing or future terminal or server, such as those applicable to the present application, shall also be included in the protection scope of the present application and are hereby included by reference.

[0056] Based on the above Figure 1In the illustrated implementation environment, the embodiment of the present application provides a training method of an image processing model, which is executed by a computer device, which can be the server 12 or the terminal 11, and the embodiment of the present application does not limit this. As shown in Figure 2 As shown, the training method of the image processing model provided by the embodiment of the present application includes the following steps 201 and 202.

[0057] In step 201, a sample image with lighting effect and a label image corresponding to the sample image without lighting effect are obtained, the sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, and the virtual light source is obtained by physical simulation on a real light source.

[0058] The sample image is an image required for training the image processing model, and the sample image is an image with lighting effect. Illustratively, the image with lighting effect refers to an image obtained by image acquisition in an environment irradiated by a light source. In addition to obtaining the sample image, the label image corresponding to the sample image without lighting effect is also obtained, so as to provide a supervision signal for the process of training the image processing model by using the label image, thereby training an image processing model capable of removing the lighting effect.

[0059] In the embodiment of the present application, the sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, wherein the virtual light source is obtained by physical simulation on a real light source. The real light source refers to a real existing light source, and the type of the real light source is not limited by the embodiment of the present application. Illustratively, the type of the real light source is a point light source, a surface light source, a line light source, etc. The virtual light source obtained by physical simulation on the real light source can simulate the irradiation of the real light source more realistically.

[0060] The virtual model of the reference object refers to a model obtained by simulating the reference object, and the reference object is an entity object. The type of the reference object is not limited by the embodiment of the present application. Illustratively, the type of the reference object is a face, an animal face, a building, etc. Illustratively, the reference object is a three-dimensional object, and the virtual model of the reference object is also a three-dimensional model.

[0061] The virtual light source can simulate the real light source more realistically, and the virtual model of the reference object can simulate the reference object more realistically. Based on this, the process of image acquisition of the virtual model of the reference object irradiated by the virtual light source can be regarded as a simulation of the process of image acquisition of the reference object irradiated by the real light source. The sample image obtained by image acquisition of the virtual model of the reference object irradiated by the virtual light source can simulate the to-be-processed image obtained by image acquisition of the reference object irradiated by the real light source more realistically, thereby facilitating improvement of the training effect of the image processing model. In addition, obtaining the sample image by image acquisition of the virtual model of the reference object irradiated by the virtual light source can avoid damage to the real light source, and the cost is relatively low.

[0062] In an example embodiment, the reference object can be an active entity object (for example, a human face), and the controllability of the virtual model of the reference object is higher than that of the reference object itself. Therefore, the embodiment of the present application can obtain a sample image with high stability based on the virtual light source and the virtual model of the reference object, thereby facilitating the training stability of the image processing model.

[0063] The label image is an image without illumination effect corresponding to the sample image. The embodiment of the present application does not limit the acquisition method of the label image. For example, the label image is obtained by manually removing the illumination effect from the sample image. For example, the label image is obtained by image acquisition of the virtual model of the reference object not irradiated by the virtual light source. The sample image and the corresponding label image are obtained by image acquisition of the virtual model of the same reference object, so as to ensure the matching degree between the sample image and the corresponding label image.

[0064] For example, in the sample image and the label image, the image features of the virtual model of the reference object itself are the same, and only the sample image has illumination effect and the label image does not have illumination effect. For example, taking the type of the reference object as a human face as an example, the sample image is as shown in (1) of Figure 3 , and the label image corresponding to the sample image is as shown in (2) of Figure 3 .

[0065] In an example embodiment, the process of obtaining the sample image and the label image in step 201 can refer to extracting the sample image and the label image from an image library, can also refer to obtaining the sample image and the label image in real time based on a virtual light source and a virtual model of a reference object, can also refer to extracting the sample image from the image library or obtaining the sample image in real time based on the virtual light source and the virtual model of the reference object, and then obtaining the label image by artificially performing the de-illumination processing on the sample image, and the like, which are not limited in the example embodiments of the present application. The image library stores an image pair composed of an image with illumination effect and an image without illumination effect obtained based on the virtual light source and the virtual model of the reference object. The image with illumination effect in the image pair is taken as the sample image, and the image without illumination effect in the image pair is taken as the label image.

[0066] It should be noted that the sample image in the example embodiments of the present application refers to an image required for training the initial image processing model once, and the number of sample images can be one or multiple, which are not limited in the example embodiments of the present application. For the case that the number of sample images is multiple, different sample images can be obtained by image acquisition on different virtual models of reference objects, or can be obtained by image acquisition on the same virtual model of the reference object, which are not limited in the example embodiments of the present application. Each sample image corresponds to a label image, and a sample image and the label image corresponding to the sample image are obtained by image acquisition on the same virtual model of the reference object, so as to ensure the model training effect. The example embodiments of the present application take the case that the number of sample images is one as an example for description.

[0067] In a possible implementation manner, the process of obtaining the sample image and the label image is as follows: loading the virtual model of the reference object into a virtual light source environment, and setting a virtual light source in the virtual light source environment; performing image acquisition on the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment to obtain the sample image; and performing image acquisition on the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment to obtain the label image.

[0068] Exemplarily, before the virtual model of the reference object is loaded into the virtual light source environment, it is necessary to construct the virtual light source environment and obtain the data of the virtual model of the reference object.

[0069] In an example embodiment, the virtual light source environment is constructed in the following manner: a virtual engine is used to set the virtual light source, and the position and size of the virtual light source are adjusted to simulate a real light source. The environment formed by the adjusted virtual light source is taken as the virtual light source environment. The virtual light source environment refers to the virtual light source environment in the virtual engine. The virtual engine refers to a virtual engine based on physical simulation. The setting of the virtual light source using the virtual engine can refer to the rendering generation of the virtual light source using the virtual engine. The virtual light source includes but is not limited to a virtual point light source, a virtual line light source, and a virtual surface light source.

[0070] In an example embodiment, the virtual engine in the embodiment of the present application refers to a UE (Unreal Engine, a game development engine). By setting the virtual light source in the UE, a real light source can be simulated more realistically. In an example embodiment, the process of adjusting the position and size of the virtual light source is implemented by a professional according to experience. By adjusting the position and size of the virtual light source, a real light source can be simulated more realistically, thereby improving the reliability of the subsequent acquired sample images.

[0071] In an example embodiment, after the virtual light source environment is constructed, one or more illumination modes can be set by a professional according to experience. The light emission of the virtual light source is different in different illumination modes, and the illumination that can be achieved is also different. The computer device can save the set illumination mode, so as to automatically control the corresponding virtual light source to illuminate according to the illumination mode subsequently.

[0072] After the virtual light source environment is constructed, the data of the virtual model of the reference object is obtained. The data of the virtual model of the reference object is obtained by scanning the virtual model of the reference object, and is used to represent the virtual model of the reference object. In an example embodiment, the virtual model of the reference object can be extracted from a virtual model database of the reference object. For example, if the type of the reference object is a face, the virtual model of the reference object can be extracted from a three-dimensional virtual model database of the face. The three-dimensional virtual model database of the face includes three-dimensional virtual models of multiple faces. In an example embodiment, the three-dimensional virtual model of the face is a virtual model obtained by three-dimensional reconstruction of a face image. In an example embodiment, the three-dimensional virtual model of the face includes a virtual model of the face part, and can also include virtual models of parts such as the neck and the head.

[0073] After the data of the virtual model of the reference object is acquired, the data of the virtual model of the reference object is loaded in the virtual engine to realize loading of the virtual model of the reference object into the virtual light source environment. After the virtual model of the reference object is loaded into the virtual light source environment, the virtual model of the reference object is placed in the virtual light source environment constructed by the virtual engine. After the virtual model of the reference object is placed in the virtual light source environment, the virtual model of the reference object can be irradiated by the virtual light source in the virtual light source environment. Illustratively, the process of irradiating the virtual model of the reference object by the virtual light source in the virtual light source environment is realized by controlling the virtual light source according to an irradiation mode. In different irradiation modes, the specific conditions of irradiating the virtual model of the reference object by the virtual light source in the virtual light source environment are different.

[0074] The sample image is obtained by image acquisition of the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment. It should be noted that in different irradiation modes, the sample image obtained by image acquisition of the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment is different, but a sample image with lighting effect can be obtained.

[0075] In addition to being able to realize irradiation of the virtual model of the reference object in different irradiation modes, the virtual model of the reference object can also not be irradiated. In the embodiment of the present application, the image obtained by image acquisition of the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment is taken as a label image. The label image obtained in this way can be regarded as an image without lighting effect corresponding to the sample image.

[0076] In the illustrative embodiment, after the virtual model of the reference object is loaded into the virtual light source environment, image acquisition can be performed on the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment in different irradiation modes to obtain a plurality of images with lighting effect. Each image in the plurality of images with lighting effect can constitute an image pair with an image without lighting effect obtained by image acquisition of the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment, and the image pair is stored in an image library to facilitate subsequent extraction of sample images and label images directly from the image library.

[0077] Exemplarily, the process of image acquisition of the virtual model of the reference object placed in the virtual light source environment can refer to the process of shooting the virtual model of the reference object by using the camera preset in the virtual engine. Exemplarily, the computer device has a display screen, and the virtual model of the reference object placed in the virtual light source environment is displayed in the display screen. In this case, the process of image acquisition of the virtual model of the reference object placed in the virtual light source environment can refer to the process of taking a screenshot of the virtual model of the reference object displayed in the display screen. That is, the virtual model of the scanned reference object is placed in the virtual light source environment, and then a high-definition and high-quality image can be obtained by taking a screenshot instead of taking a photo, so as to realize the process of manufacturing the training data of the image processing model for light removal.

[0078] The virtual light source environment is constructed by using the virtual engine, and the virtual model of the reference object placed in the virtual light source environment is relit and rendered, and then a screenshot is taken to simulate the photo shooting under the real light source. On the basis of the virtual model of the reference object, a batch of paired training data (images with light effects and images without light effects) can be manufactured by relighting and rendering the virtual model of the scanned reference object in the virtual engine, and then the training of the image processing model can be realized. This method of manufacturing training data has low cost and can effectively simulate the illumination of the real light source.

[0079] In step 202, the initial image processing model is trained by using the sample image and the label image to obtain a target image processing model, and the target image processing model is used for light removal processing of the image to be processed with light effects, and the image to be processed with light effects is obtained by image acquisition of the reference object irradiated by the real light source.

[0080] After obtaining the sample image and the label image, the initial image processing model is trained by using the sample image and the label image to obtain a target image processing model trained for light removal processing of the image to be processed without light effects, and the image to be processed without light effects is obtained by image acquisition of the reference object irradiated by the real light source.

[0081] The initial image processing model refers to an image processing model to be trained by using the sample image and the label image. The initial image processing model is a model capable of converting one image into another image. Exemplarily, the initial image processing model refers to an image translation model (or an image conversion model), such as a pix2pix model (an image translation model), a pix2pixHD model (an image translation model), etc.

[0082] The model structure of the initial image processing model is not limited in the embodiments of the present application, as long as the initial image processing model can have the function of performing the de-illumination processing on the input image. Illustratively, the initial image processing model includes a generator and a discriminator, wherein the generator is configured to perform the de-illumination processing on the input image to output another image; and the discriminator is configured to distinguish whether the input image is the image output by the generator or a real image. The structure of the generator and the discriminator is not limited in the embodiments of the present application. Illustratively, the initial image processing model itself is a generator.

[0083] The training of the initial image processing model by using the sample image and the label image can mean that the initial image processing model is directly trained by using the sample image and the label image, or can mean that the initial image processing model is trained by using the first texture image corresponding to the sample image and the second texture image corresponding to the label image. The embodiments of the present application are not limited in this regard. The implementation manner of the step 202 is different in different cases.

[0084] In a possible implementation manner, for the case that the initial image processing model is trained by using the first texture image corresponding to the sample image and the second texture image corresponding to the label image, the implementation manner of the step 202 includes the step 2021 to the step 2023.

[0085] The step 2021 acquires the first texture image corresponding to the sample image and the second texture image corresponding to the label image.

[0086] The first texture image is used to characterize the surface texture of the virtual model of the reference object on which the sample image is acquired, and the second texture image is used to characterize the surface texture of the virtual model of the reference object on which the label image is acquired. Illustratively, the first texture image is a texture image with illumination effect, and the second texture image is a texture image without illumination effect. The second texture image can be regarded as an image without illumination effect corresponding to the first texture image. Therefore, the first texture image and the second texture image can be used to train an image processing model for removing the illumination effect.

[0087] Illustratively, taking the type of the reference object as a face as an example, the first texture image is used to characterize the skin texture of the virtual model of the face irradiated by the virtual light source, and the second texture image is used to characterize the skin texture of the virtual model of the face not irradiated by the virtual light source.

[0088] In a possible implementation manner, the process of obtaining the first texture image corresponding to the sample image refers to a process of converting a texture from a sample image space to a texture space based on a position relationship between a virtual model point of a virtual model constituting a reference object and the texture space. Exemplarily, the texture image mentioned in the embodiments of the present application is a UV (the names of two coordinate axes of a texture coordinate system referred to by a texture space) texture image, the texture space refers to a UV space, and the process of obtaining the first texture image corresponding to the sample image can be regarded as a process of unfolding the sample image into a UV texture image.

[0089] In an exemplary embodiment, the position relationship between the virtual model point of the virtual model constituting the reference object and the texture space is determined by a professional according to a pre-set virtual model topology structure, which can indicate which position in the texture space a virtual model point of each virtual model constituting the virtual model corresponds to, for example, a first virtual model point constituting the virtual model corresponds to a point at a (0, 0) position in the texture space, a second virtual model point constituting the virtual model corresponds to a point at a (0, 1) position in the texture space, and the like.

[0090] Exemplarily, the virtual models of the same type of reference object are constituted by the same number of virtual model points, and the position relationship between the virtual model points constituting the virtual models of the same type of reference object and the texture space is fixed and unchangeable. The number of virtual model points constituting the virtual models of different types of reference objects can be the same or different, which is not limited in the embodiments of the present application. Exemplarily, the embodiments of the present application do not limit which virtual model points constitute the virtual model, and the virtual model can be flexibly set according to the type of the reference object.

[0091] Exemplarily, the process of obtaining the first texture image corresponding to the sample image is as follows: determining a texture point corresponding to the virtual model point in the texture space and a mapping point corresponding to the virtual model point in the sample image, the virtual model point being a point of a virtual model constituting a reference object; determining a pixel value of the texture point based on a pixel value of the mapping point in the sample image; and obtaining the first texture image based on the texture point and the pixel value of the texture point.

[0092] The texture point corresponding to the virtual model point in the texture space is determined based on the positional relationship between the virtual model point and the point in the texture space. Since the sample image is an image obtained by image acquisition of the virtual model illuminated by the virtual light source, the virtual model can be mapped onto the sample image in the image acquisition direction. The mapping point corresponding to the virtual model point in the sample image can be determined through the mapping. After the texture point and the mapping point are determined, the pixel value possessed by the texture point is determined based on the pixel value possessed by the mapping point in the sample image. The pixel value possessed by the mapping point in the sample image is used to represent the texture feature of the virtual model point itself. The pixel value possessed by the texture point is determined based on the pixel value possessed by the mapping point in the sample image, so that the process of converting the texture feature from the sample image space to the texture space can be realized.

[0093] Exemplarily, the number of virtual model points of the virtual model constituting the reference object is a plurality, each virtual model point corresponds to a texture point and a mapping point, and the texture point and the mapping point corresponding to the same virtual model point have a corresponding relationship. Exemplarily, the implementation process of determining the pixel value possessed by the texture point based on the pixel value possessed by the mapping point in the sample image is as follows: for any one of the mapping points, the texture point corresponding to the any one of the mapping points is determined among the texture points, and the pixel value possessed by the any one of the mapping points in the sample image is taken as the pixel value possessed by the texture point corresponding to the any one of the mapping points. Based on this mode, after all the mapping points are traversed, the pixel values possessed by all the texture points can be determined.

[0094] After the texture point and the pixel value possessed by the texture point are determined, the first texture image is obtained based on the texture point and the pixel value possessed by the texture point. Exemplarily, the number of virtual model points of the virtual model constituting the reference object is a plurality, each virtual model point corresponds to a texture point and a mapping point, and the first texture image is obtained based on all the texture points and the pixel values possessed by all the texture points. Exemplarily, different texture points are located at different positions, and the region between the texture points is interpolated using the pixel values possessed by the texture points, so that the first texture image can be obtained. Exemplarily, taking the type of the reference object as a face as an example, if the sample image is as shown in (1) of FIG. 2, Figure 4 , the first texture image corresponding to the sample image is as shown in (2) of FIG. 2. Figure 4 .

[0095] Exemplarily, the process of obtaining the second texture image corresponding to the label image is as follows: the texture point corresponding to the virtual model point in the texture space and the mapping point corresponding to the virtual model point in the label image are determined; the pixel value possessed by the texture point is determined based on the pixel value possessed by the mapping point in the label image; and the second texture image is obtained based on the texture point and the pixel value possessed by the texture point. The implementation principle of this process is the same as that of the process of obtaining the first texture image corresponding to the sample image, which will not be described herein again.

[0096] Step 2022: input the first texture image into the initial image processing model for de-lighting processing to obtain a third texture image output by the initial image processing model.

[0097] After the first texture image is acquired, the first texture image is input into the initial image processing model, and the initial image processing model performs de-lighting processing on the first texture image to obtain a third texture image output by the initial image processing model. The process of de-lighting processing of the first texture image by the initial image processing model is an internal processing process of the initial image and is related to the structure of the initial image processing model, which is not limited in the embodiments of the present application. The third texture image is an image of the de-lighting effect corresponding to the first texture image predicted by the initial image processing model. For example, the first texture image with the lighting effect shown in (1) of FIG. 14 is input into the initial image processing model for de-lighting processing, and the third texture image with the de-lighting effect shown in (2) of FIG. 14 can be obtained. Figure 5 Figure 5

[0098] The processing process of inputting the first texture image into the initial image processing model for de-lighting processing is related to the structure of the initial image processing model, which is not limited in the embodiments of the present application. In an exemplary embodiment, the initial image processing model includes a generator and a discriminator, and the implementation of step 2022 is that the first texture image is input into the generator in the initial image processing model for de-lighting processing to obtain a third texture image output by the generator in the initial image processing model. In an exemplary embodiment, the initial image processing model itself is a generator, and the implementation of step 2022 is that the first texture image is directly input into the initial image processing model for de-lighting processing to obtain a third texture image output by the initial image processing model.

[0099] Step 2023: based on the third texture image and the second texture image, a first loss function is acquired; and the initial image processing model is trained by using the first loss function.

[0100] The third texture image is an image of the de-lighting effect corresponding to the first texture image predicted by the initial image processing model, and the second texture image is a standard image of the de-lighting effect corresponding to the first texture image. Based on the third texture image and the second texture image, a first loss function providing a supervision signal for the training process of the initial image processing model can be acquired.

[0101] ​​The manner of obtaining the first loss function based on the third texture image and the second texture image is related to the structure of the initial image processing model, and embodiments of the present application do not limit this. In a possible implementation manner, the initial image processing model includes a generator and a discriminator, and the third texture image is obtained by inputting the first texture image into the generator in the initial image processing model for de-illumination processing. In this case, based on the third texture image and the second texture image, the process of obtaining the first loss function includes the following steps 1 to 3.

[0102] Step 1: input the second texture image into the discriminator in the initial image processing model for discrimination to obtain a first discrimination result output by the discriminator in the initial image processing model, the first discrimination result being used to indicate the probability that the second texture image is an image output by the generator; input the third texture image into the discriminator in the initial image processing model for discrimination to obtain a second discrimination result output by the discriminator in the initial image processing model, the second discrimination result being used to indicate the probability that the third texture image is an image output by the generator.

[0103] The discriminator in the initial image processing model is used to discriminate whether an input image is an image output by the generator or a real image. After inputting the second texture image and the third texture image into the discriminator in the initial image processing model, the first discrimination result and the second discrimination result output by the discriminator in the initial image processing model can be obtained. The first discrimination result is used to indicate the probability that the second texture image is an image output by the generator, and the second discrimination result is used to indicate the probability that the third texture image is an image output by the generator.

[0104] Embodiments of the present application do not limit the representation form of the discrimination result output by the discriminator in the initial image processing model, as long as it can indicate the probability that the input image is an image output by the generator. For example, the representation form of the discrimination result is a probability with a value range of 0-1, which can represent the probability that the input image is an image output by the generator, or the probability that the input image is a real image, and embodiments of the present application do not limit this. If the probability represents the probability that the input image is a real image, the difference between 1 and the probability can be used as the probability that the input image is an image output by the generator. For example, taking the probability that the input image is a real image as an example, if the discrimination result is 1, it means that the discriminator considers that the input image is a real image, and if the discrimination result is 0, it means that the discriminator considers that the input image is an image output by the generator. For example, the representation form of the discrimination result can also be a probability group, one probability in the probability group representing the probability that the input image is a real image, and the other probability representing the probability that the input image is an image output by the generator.

[0105] Step 2: obtaining a first sub-loss function based on the first discrimination result and the second discrimination result.

[0106] After obtaining the first discrimination result and the second discrimination result, a generative adversarial loss function is obtained based on the first discrimination result and the second discrimination result, and the generative adversarial loss function is taken as the first sub-loss function. Illustratively, the first sub-loss function is calculated based on formula 1:

[0107] L_gan = E x [log(D(x)] + E z [log(1-D(G(z)))] (formula 1)

[0108] Wherein, L_gan represents the first sub-loss function; x represents the second texture image; D(x) represents the first discrimination result; z represents the first texture image; G(z) represents the third texture image; D(G(z)) represents the second discrimination result; E represents mathematical expectation. Illustratively, the representation form of the discrimination result used in the formula 1 is a probability that an input image is a real image.

[0109] Step 3: obtaining a first loss function based on the first sub-loss function.

[0110] After obtaining the first sub-loss function, the first loss function is obtained based on the first sub-loss function. In an illustrative embodiment, for the case that the initial image processing model includes a generator and a discriminator, the process of training the initial image processing model is a process of alternately training the generator and the discriminator. According to whether the current training stage is the stage of training the generator or the stage of training the discriminator, the way of obtaining the first loss function based on the first sub-loss function is different.

[0111] In an illustrative embodiment, for the case that the current training stage is the stage of training the discriminator, the way of obtaining the first loss function based on the first sub-loss function is that the first sub-loss function is taken as the first loss function.

[0112] In an illustrative embodiment, for the case that the current training stage is the stage of training the generator, the way of obtaining the first loss function based on the first sub-loss function includes the following steps a and step b.

[0113] Step a: obtaining a second sub-loss function based on the second texture image and the third texture image.

[0114] In a possible implementation, the second sub-loss function is directly obtained based on the second texture image and the third texture image, for example, an L1 loss function between the second texture image and the third texture image is taken as the second sub-loss function, or a cross-entropy loss function between the second texture image and the third texture image is taken as the second sub-loss function. For example, the second texture image and the third texture image have the same size, and the loss function between the second texture image and the third texture image is calculated based on a difference between a pixel value of a pixel point in the second texture image and a pixel value of a pixel point at a corresponding position in the third texture image.

[0115] For example, for a case where the second sub-loss function is an L1 loss function between the second texture image and the third texture image, the second sub-loss function is calculated based on formula 2.

[0116] L_L1=|I_gt–I_pred| (formula 2)

[0117] Wherein, L_L1 represents the second sub-loss function, I_gt represents the second texture image, and I_pred represents the third texture image.

[0118] In another possible implementation, the second sub-loss function is obtained by determining a first sub-image in the second texture image that satisfies a reliable condition and a second sub-image in the third texture image that satisfies the reliable condition, and obtaining the second sub-loss function based on the first sub-image and the second sub-image.

[0119] The first sub-image is a part of the second texture image that satisfies the reliable condition, the second sub-image is a part of the third texture image that satisfies the reliable condition, and the first sub-image and the second sub-image are images on which the second sub-loss function is calculated. The reliable condition is set according to experience or is flexibly adjusted according to an application scenario, and embodiments of the present application do not limit this. For example, the reliable condition means that a pixel value of a pixel point included is within a reference value range, and the reference value range is set according to experience or is flexibly adjusted according to an application scenario, and embodiments of the present application do not limit this.

[0120] In an example embodiment, the reliable condition means that a position is the same as a position of a reference region in a mask image. The mask image is used to assist in determining a sub-image in a texture image for calculating a loss function. The mask image includes the reference region, and the position of the reference region in the mask image is used to constrain a position of the sub-image in the texture image for calculating the loss function. For example, the position of the reference region in the mask image is set according to experience or is flexibly adjusted according to specific circumstances of a virtual model of a reference object, and embodiments of the present application do not limit this.

[0121] In an example embodiment, the mask image is determined according to a virtual model of the reference object on which the sample image is acquired, the mask image has the same size as the texture image, and the reference region in the mask image refers to a region in which a point corresponding to a virtual model point visible on the front of the virtual model of the reference object on which the sample image is acquired is located in the texture space. In this way, the position of the reference region in the mask image is a position with a relatively high degree of credibility. For example, the reference region and other regions in the mask image are marked with different colors to facilitate quick differentiation between the reference region and other regions, for example, the reference region in the mask image is marked with white color and other regions are marked with black color. For example, the mask image is as shown in FIG. 8. Figure 6 Figure 6 The white region in the mask image is the reference region, and the black region is the other region.

[0122] After the mask image is determined, a sub-image in the second texture image having the same position as the position of the reference region in the mask image is taken as the first sub-image, and a sub-image in the third texture image having the same position as the position of the reference region in the mask image is taken as the second sub-image. For example, since the second texture image and the third texture image have the same size, the first sub-image and the second sub-image also have the same size.

[0123] After the first sub-image and the second sub-image are acquired, a second sub-loss function is acquired based on the first sub-image and the second sub-image. The way in which the second sub-loss function is acquired based on the first sub-image and the second sub-image is not limited in the embodiments of the present application. For example, an L1 loss function between the first sub-image and the second sub-image is taken as the second sub-loss function; or a cross-entropy loss function between the first sub-image and the second sub-image is taken as the second sub-loss function.

[0124] Step b: acquiring a first loss function based on the first sub-loss function and the second sub-loss function.

[0125] After the first sub-loss function and the second sub-loss function are acquired, a product of the first sub-loss function and a first weight and a product of the second sub-loss function and a second weight are calculated, and a sum of the two products is taken as the first loss function. The first weight and the second weight are set according to experience or flexibly adjusted according to an application scenario, which is not limited in the embodiments of the present application. For example, the first weight is 1 and the second weight is 100. For example, the process of acquiring the first loss function based on the first sub-loss function and the second sub-loss function is calculated based on formula 3:

[0126] L=L_L1*w_L1+L_gan*w_gan (formula 3)

[0127] ​wherein, L represents the first loss function; L L1 represents the second sub-loss function; L_gan represents the first sub-loss function; w L1 represents the second weight; and w_gan represents the first weight.

[0128] It should be noted that the above only takes the initial image processing model including the generator and the discriminator as an example to introduce the process of obtaining the first loss function based on the third texture image and the second texture image. The embodiments of the present application are not limited thereto. In the exemplary embodiments, the initial image processing model can be a generator itself. In this case, the process of obtaining the first loss function based on the third texture image and the second texture image is to directly take the L1 loss function or the cross-entropy loss function between the third texture image and the second texture image as the first loss function.

[0129] After obtaining the first loss function, the initial image processing model is trained by using the first loss function. The manner of training the initial image processing model by using the first loss function is related to the structure of the initial image processing model. In the exemplary embodiments, for the case that the initial image processing model is a generator itself, the manner of training the initial image processing model by using the first loss function is to update the parameters of the initial image processing model with the optimization objective of minimizing the first loss function.

[0130] In the exemplary embodiments, for the case that the initial image processing model includes the generator and the discriminator, the manner of training the initial image processing model by using the first loss function is related to the current training stage. Exemplarily, if the current training stage is the stage of training the discriminator, the manner of training the initial image processing model by using the first loss function is to update the parameters of the discriminator in the initial image processing model with the optimization objective of maximizing the first loss function while keeping the parameters of the generator in the initial image processing model unchanged; if the current training stage is the stage of training the generator, the manner of training the initial image processing model by using the first loss function is to update the parameters of the generator in the initial image processing model with the optimization objective of minimizing the first loss function while keeping the parameters of the discriminator in the initial image processing model unchanged.

[0131] After training the initial image processing model once by using the first loss function, it is determined whether the training process meets the termination condition. If the termination condition is not met, the first loss function is obtained based on the initial image processing model after being trained once, and the initial image processing model after being trained once is trained once by using the first loss function. The above process is repeated until the training process meets the termination condition. The target image processing model is obtained based on the image processing model obtained when the termination condition is met.

[0132] Exemplarily, for the case that the initial image processing model itself is a generator, satisfying the termination condition includes but is not limited to any one of the following: the number of training reaches a first number threshold, the first loss function converges, and the first loss function is less than a first loss threshold.

[0133] Exemplarily, for the case that the initial image processing model includes a generator and a discriminator, satisfying the termination condition includes but is not limited to at least one of the following: the number of alternately training the generator and the discriminator is not less than a second number threshold, and the generative adversarial loss function converges.

[0134] In an exemplary embodiment, the sample image and the label image are first expanded by UV to obtain a first texture image and a second texture image, and then the first texture image and the second texture image are used to train the image processing model. Taking the type of the reference object as a human face as an example, this way can ensure that the model generates a texture image that can represent a complete human face skin, rather than a human face image.

[0135] Exemplarily, in the process of training the initial image processing model using the texture image, the input is a UV texture image with lighting effect, and the output is an albedo (reflectivity) UV texture image without lighting. For the case of calculating the second sub-loss function based on the sub-image in the texture image and the reference region in the mask image, since the missing parts of the surface texture of different reference objects are different, after training, the image processing model can learn to complete and repair the missing parts of the texture image through data, thereby being able to repair an incomplete texture image to a certain extent.

[0136] In a possible implementation manner, for the case of training the initial image processing model using the sample image and the label image directly, the implementation manner of the step 202 includes a step 202A and a step 202B.

[0137] Step 202A: inputting the sample image into the initial image processing model for de-lighting processing to obtain a predicted image output by the initial image processing model.

[0138] The principle of inputting the sample image into the initial image processing model for de-lighting processing to obtain a predicted image output by the initial image processing model is the same as that of inputting the first texture image into the initial image processing model for de-lighting processing to obtain a third texture image output by the initial image processing model. For details, refer to the step 2022, which will not be described here.

[0139] Step 202B: obtaining a second loss function based on the predicted image and the label image; and training the initial image processing model using the second loss function.

[0140] The principle of obtaining the second loss function based on the predicted image and the label image is the same as the principle of obtaining the first loss function based on the third texture image and the second texture image, and the principle of training the initial image processing model by using the second loss function is the same as the principle of training the initial image by using the first loss function. For the implementation of the step 202B, refer to the step 2023, and details are not repeated here.

[0141] After obtaining the target image processing model, the target image post-processing model is called to perform the de-lighting processing on the to-be-processed image with the lighting effect. For the implementation of the process, refer to the embodiment shown in Figure 7 , and details are not repeated here.

[0142] The embodiment of the present application can obtain training data in a low-cost manner, and train an image processing model to complete the work of removing the highlight shadow of a face, so that the appearance of a face without lighting can be obtained for any photo.

[0143] The training method of the image processing model provided by the embodiment of the present application is that the sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source. The sample image is obtained without irradiation of a real light source, and the cost is low. In addition, the virtual light source obtained by physical simulation of the real light source can simulate the real light source more realistically, and the virtual model of the reference object can simulate the reference object more realistically. Therefore, the sample image with the lighting effect can simulate the to-be-processed image with the lighting effect more realistically. The effect of training the image processing model by using the sample image and the corresponding label image is good, which is beneficial to improve the reliability of the de-lighting processing of the to-be-processed image with the lighting effect by using the trained image processing model, and the image processing quality is high.

[0144] Based on the implementation environment shown in Figure 1 , the embodiment of the present application provides an image processing method. The training method of the image processing model is executed by a computer device. The computer device can be a server 12 or a terminal 11, and the embodiment of the present application does not limit this. As shown in Figure 7 , the image processing method provided by the embodiment of the present application includes the following steps 701 and 702.

[0145] In the step 701, a to-be-processed image with a lighting effect and a target image processing model are obtained. The target image processing model is trained by using a sample image with a lighting effect and a label image without a lighting effect corresponding to the sample image. The sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source. The virtual light source is obtained by physical simulation of a real light source.

[0146] The to-be-processed image is obtained by image acquisition on a reference object irradiated by a real light source.

[0147] The real light source is in a real light source environment. For example, the to-be-processed image is obtained by image acquisition on a reference object placed in the real light source environment and irradiated by a real light source in the real light source environment. For example, the reference object irradiated by the real light source is photographed by using a camera. The to-be-processed image with the lighting effect in step 701 can be obtained by extracting a pre-acquired and stored to-be-processed image, or can be obtained by photographing, which is not limited in the embodiments of the present application.

[0148] The target image processing model is obtained by training using a sample image with a lighting effect and a label image without a lighting effect corresponding to the sample image. The sample image is obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source. The virtual light source is obtained by physical simulation on a real light source. The target image processing model in step 701 can be obtained by extracting a pre-trained and stored target image processing model, or can be obtained by real-time training, which is not limited in the embodiments of the present application. The process of training the target image processing model is described in detail in the embodiments of the present application. Figure 2 The embodiments are not described herein again.

[0149] In step 702, the target image processing model is called to perform de-lighting processing on the to-be-processed image to obtain a target image.

[0150] The target image processing model has the function of performing de-lighting processing on an image with a lighting effect. After the to-be-processed image with the lighting effect and the target image processing model are obtained, the target image processing model is called to perform de-lighting processing on the to-be-processed image. The image obtained by calling the target image processing model to perform de-lighting processing on the to-be-processed image is taken as the target image. The target image is an image without a lighting effect.

[0151] In the exemplary embodiments, according to different ways of training the target image processing model, the way of calling the target image processing model to perform de-lighting processing on the to-be-processed image to obtain the target image is different.

[0152] In the exemplary embodiments, the target image processing model is obtained by training an initial image processing model using a first texture image corresponding to the sample image and a second texture image corresponding to the label image (the training process is described in detail in the embodiments of the present application). Figure 2In the step 2021 to the step 2023 of the embodiment shown, in this case, the target image processing model is called to perform the de-lighting processing on the to-be-processed image, and the manner of obtaining the target image is that a texture image corresponding to the to-be-processed image is obtained; the texture image corresponding to the to-be-processed image is input into the target image processing model to perform the de-lighting processing, and a target image output by the target image processing model is obtained. In this case, the target image obtained is an image with a de-lighting effect corresponding to the texture image, and is essentially a texture image.

[0153] For example, the manner of obtaining the texture image corresponding to the to-be-processed image is that a virtual model of the reference object is reconstructed according to the to-be-processed image; a texture point corresponding to a virtual model point in a texture space and a mapping point corresponding to the virtual model point in the to-be-processed image are determined, the virtual model point being a point constituting the virtual model of the reference object reconstructed; a pixel value of the texture point is determined based on a pixel value of the mapping point in the to-be-processed image; and the texture image corresponding to the to-be-processed image is obtained based on the texture point and the pixel value of the texture point. The to-be-processed image is an image obtained by image acquisition on the reference object, and the virtual model of the reference object on which the to-be-processed image is based can be reconstructed according to the to-be-processed image, and then the process of converting the to-be-processed image into the texture image is realized according to the virtual model of the reference object reconstructed.

[0154] In the example embodiment, the target image processing model is obtained by directly training the initial image processing model by using the sample image and the label image (for details of the training process, please refer to the description of the step 201A and the step 201B of the embodiment shown in FIG. 2). Figure 2 In the step 202A and the step 202B of the embodiment shown, in this case, the target image processing model is called to perform the de-lighting processing on the to-be-processed image, and the manner of obtaining the target image is that the to-be-processed image is input into the target image processing model to perform the de-lighting processing, and a target image output by the target image processing model is obtained. In this case, the target image obtained is an image with a de-lighting effect corresponding to the to-be-processed image, and is essentially an image of the reference object.

[0155] For example, the final purpose of the image processing is to obtain a high-definition and flat UV texture image. After the to-be-processed image is input into the target image processing model to perform the de-lighting processing, and a target image output by the target image processing model is obtained, the process further includes obtaining a texture image corresponding to the target image. The principle of obtaining the texture image corresponding to the target image is the same as the principle of obtaining the texture image corresponding to the to-be-processed image, which will not be described herein again.

[0156] In an exemplary embodiment, after obtaining the texture image corresponding to the target image or obtaining a target image that is essentially a texture image, the texture image can be post-processed to optimize it. The post-processed texture image is used to characterize the surface texture of the reference object. The post-processed texture image can be used to apply textures to the model to be rendered, thereby rendering a virtual model with a surface texture characterized by the texture image.

[0157] The image processing method provided in this application can be used for photo enhancement, facial texture calculation, etc. It can achieve image delighting processes with higher quality and lower cost than related technologies. For example, taking a texture image as the input target image processing model, the method provided in this application can enable... Figure 8 The textured image shown in (1) becomes like... Figure 8 The image shown in (2) is a flat texture image, but facial features are not lost.

[0158] The image processing method provided in this application uses a target image processing model trained with sample images with lighting effects and corresponding label images without lighting effects. The sample images are obtained by acquiring images of a virtual model of a reference object illuminated by a virtual light source. This acquisition process does not require real light source illumination, resulting in lower costs. Furthermore, the virtual light source obtained through physical simulation of a real light source can realistically simulate the real light source, and the virtual model of the reference object can realistically simulate the reference object. Therefore, the sample images with lighting effects can realistically simulate the image to be processed with lighting effects. Training the image processing model using these sample images and corresponding label images yields good results. The trained target image processing model has high reliability in removing lighting effects from the image to be processed, resulting in high image processing quality.

[0159] See Figure 9 This application provides a training apparatus for an image processing model, the apparatus comprising:

[0160] The acquisition unit 901 is used to acquire a sample image with lighting effects and a label image without lighting effects corresponding to the sample image. The sample image is obtained by image acquisition of a virtual model of a reference object illuminated by a virtual light source. The virtual light source is obtained by physical simulation of a real light source.

[0161] The training unit 902 is configured to train the initial image processing model by using the sample image and the label image, to obtain a target image processing model, and the target image processing model is configured to perform de-lighting processing on the to-be-processed image with lighting effects, and the to-be-processed image is obtained by image acquisition on the reference object irradiated by the real light source.

[0162] In a possible implementation, the acquisition unit 901 is configured to load a virtual model of the reference object into a virtual light source environment, and the virtual light source environment is provided with a virtual light source; perform image acquisition on the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment, to obtain a sample image; and perform image acquisition on the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment, to obtain a label image.

[0163] In a possible implementation, the training unit 902 is configured to obtain a first texture image corresponding to the sample image and a second texture image corresponding to the label image; input the first texture image into the initial image processing model to perform de-lighting processing, to obtain a third texture image output by the initial image processing model; obtain a first loss function based on the third texture image and the second texture image; and train the initial image processing model by using the first loss function.

[0164] In a possible implementation, the initial image processing model includes a generator and a discriminator, and the third texture image is obtained by inputting the first texture image into the generator in the initial image processing model to perform de-lighting processing; the training unit 902 is configured to input the second texture image into the discriminator in the initial image processing model to perform discrimination, to obtain a first discrimination result output by the discriminator in the initial image processing model, and the first discrimination result is configured to indicate a probability that the second texture image is an image output by the generator; input the third texture image into the discriminator in the initial image processing model to perform discrimination, to obtain a second discrimination result output by the discriminator in the initial image processing model, and the second discrimination result is configured to indicate a probability that the third texture image is an image output by the generator; obtain a first sub-loss function based on the first discrimination result and the second discrimination result; and obtain the first loss function based on the first sub-loss function.

[0165] In a possible implementation, the training unit 902 is configured to determine a first sub-image in the second texture image satisfying a reliable condition and a second sub-image in the third texture image satisfying the reliable condition; obtain a second sub-loss function based on the first sub-image and the second sub-image; and obtain the first loss function based on the first sub-loss function and the second sub-loss function.

[0166] In a possible implementation, the training unit 902 is configured to determine a texture point corresponding to the virtual model point in the texture space and a mapping point corresponding to the virtual model point in the sample image, the virtual model point being a point of a virtual model of the reference object; determine a pixel value possessed by the texture point based on a pixel value possessed by the mapping point in the sample image; and obtain the first texture image based on the texture point and the pixel value possessed by the texture point.

[0167] In a possible implementation, the training unit 902 is configured to input the sample image into an initial image processing model to perform de-lighting processing, to obtain a predicted image output by the initial image processing model; obtain a second loss function based on the predicted image and the label image; and train the initial image processing model by using the second loss function.

[0168] The training apparatus for the image processing model provided in the embodiments of the present application has the following advantages. The sample image is obtained by image acquisition on the virtual model of the reference object irradiated by the virtual light source, and the sample image is obtained without irradiation of the real light source, so the cost is low. In addition, the virtual light source obtained by physical simulation of the real light source can simulate the real light source more realistically, and the virtual model of the reference object can simulate the reference object more realistically, so the sample image with the lighting effect can simulate the to-be-processed image with the lighting effect more realistically. Therefore, the image processing model is trained by using the sample image and the corresponding label image, and the effect is good, which is beneficial to improving the reliability of de-lighting processing of the to-be-processed image with the lighting effect by using the trained image processing model, and the image processing quality is high.

[0169] Referring to Figure 10 The embodiments of the present application provide an image processing apparatus, which comprises:

[0170] The obtaining unit 1001 is configured to obtain a to-be-processed image with a lighting effect and a target image processing model, the target image processing model being trained by using a sample image with the lighting effect and a label image without the lighting effect corresponding to the sample image, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source being obtained by physical simulation of a real light source, and the to-be-processed image being obtained by image acquisition on the reference object irradiated by the real light source.

[0171] The processing unit 1002 is configured to call the target image processing model to perform de-lighting processing on the to-be-processed image, to obtain a target image.

[0172] In a possible implementation, the processing unit 1002 is configured to obtain a texture image corresponding to the to-be-processed image; and input the texture image corresponding to the to-be-processed image into the target image processing model to perform de-lighting processing, to obtain the target image output by the target image processing model.

[0173] In a possible implementation, the processing unit 1002 is configured to input the image to be processed into the target image processing model to perform de-illumination processing, and obtain a target image output by the target image processing model.

[0174] The image processing apparatus provided in the embodiments of the present application is configured to train the target image processing model by using the sample image with the illumination effect and the corresponding label image without the illumination effect, wherein the sample image is obtained by performing image acquisition on the virtual model of the reference object irradiated by the virtual light source. The sample image is obtained without irradiation of the real light source, and thus the cost is low. In addition, the virtual light source obtained by physically simulating the real light source can simulate the real light source more realistically, and the virtual model of the reference object can simulate the reference object more realistically. Therefore, the sample image with the illumination effect can simulate the image to be processed with the illumination effect more realistically. The image processing model is trained by using the sample image and the corresponding label image, and the effect is good. The target image processing model is trained, and the image to be processed with the illumination effect is de-illuminated, and the reliability is high. The image processing quality is high.

[0175] It should be noted that the apparatus provided in the above embodiments is only used as an example to illustrate the division of the above functional units. In actual applications, the above functions can be completed by different functional units according to needs, that is, the internal structure of the device is divided into different functional units to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be described here.

[0176] In the example embodiments, a computer device is also provided, which includes a processor and a memory. The memory stores at least one computer program. The at least one computer program is loaded and executed by the one or more processors, so that the computer device implements the training method of any one of the above image processing models or the image processing method. The computer device can be a server or a terminal, and the embodiments of the present application do not limit this. Next, the structure of the server and the terminal will be introduced respectively.

[0177] Figure 11Fig. 1 is a schematic diagram of a server according to an embodiment of the present application. The server can have a large difference due to different configurations or performances, and can include one or more processors (Central Processing Units, CPUs) 1101 and one or more memories 1102. The one or more memories 1102 store at least one computer program, which is loaded and executed by the one or more processors 1101, so that the server implements the training method of the image processing model or the image processing method provided by any of the above method embodiments. Of course, the server can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for realizing the functions of the device, and will not be described here.

[0178] Figure 12 Fig. 2 is a schematic diagram of a terminal according to an embodiment of the present application. The terminal can be, for example, a PC, a mobile phone, a smartphone, a PDA, a wearable device, a PPC, a tablet computer, a smart car, a smart television, a smart speaker, a smart voice interaction device, a smart home appliance, a vehicle terminal, etc. The terminal can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0179] Generally, the terminal includes a processor 1201 and a memory 1202.

[0180] The processor 1201 can include one or more processing cores. The processor 1201 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU. The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1201 can be integrated with a GPU, which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 1201 can also include an AI processor for processing machine learning-related computing operations.

[0181] The memory 1202 can include one or more computer-readable storage media, which can be non-transitory. The memory 1202 can also include a high-speed random access memory and a non-volatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1202 is used to store at least one instruction for being executed by the processor 1201, so that the terminal implements the training method of the image processing model or the image processing method provided by the method embodiments of the present application.

[0182] In some embodiments, the terminal can further optionally include a peripheral device interface 1203 and at least one peripheral device. The processor 1201, the memory 1202 and the peripheral device interface 1203 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1203 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 1204, a display screen 1205, a camera component 1206, an audio circuit 1207 and a power supply 1209.

[0183] The peripheral device interface 1203 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1201 and the memory 1202. The radio frequency circuit 1204 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1204 communicates with communication networks and other communication devices through electromagnetic signals. The display screen 1205 is used to display a UI (User Interface). The UI can include graphics, text, icons, videos and any combination thereof. The camera component 1206 is used to capture images or videos.

[0184] The audio circuit 1207 can include a microphone and a speaker. The microphone is used to capture sound waves of a user and an environment and convert the sound waves into an electrical signal input to the processor 1201 for processing or input to the radio frequency circuit 1204 to achieve voice communication. The speaker is used to convert an electrical signal from the processor 1201 or the radio frequency circuit 1204 into sound waves. The power supply 1209 is used to supply power to each component in the terminal. The power supply 1209 can be alternating current, direct current, disposable batteries or rechargeable batteries.

[0185] In some embodiments, the terminal further includes one or more sensors 1210. The one or more sensors 1210 include, but are not limited to, an acceleration sensor 1211, a gyroscope sensor 1212, a pressure sensor 1213, an optical sensor 1215 and a proximity sensor 1216.

[0186] The acceleration sensor 1211 can detect the acceleration magnitude in three coordinate axes of a coordinate system established by the terminal. The gyro sensor 1212 can detect the body direction and rotation angle of the terminal, and the gyro sensor 1212 can cooperate with the acceleration sensor 1211 to collect the 3D action of the user on the terminal. The pressure sensor 1213 can be arranged on the side frame of the terminal and / or the lower layer of the display screen 1205. When the pressure sensor 1213 is arranged on the side frame of the terminal, the holding signal of the user on the terminal can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 1201 according to the holding signal collected by the pressure sensor 1213. When the pressure sensor 1213 is arranged on the lower layer of the display screen 1205, the operable control on the UI interface can be controlled by the processor 1201 according to the pressure operation of the user on the display screen 1205.

[0187] The optical sensor 1215 is used to collect the ambient light intensity. The proximity sensor 1216, also called the distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 1216 is used to collect the distance between the user and the front of the terminal.

[0188] Those skilled in the art can understand that Figure 12 The structure shown in the foregoing embodiments does not constitute a limitation on the terminal, and the terminal can include more or fewer components than those shown in the drawings, or combine certain components, or adopt a different component arrangement.

[0189] In an example embodiment, a computer readable storage medium is also provided, and the computer readable storage medium stores at least one computer program. The at least one computer program is loaded and executed by a processor of a computer device, so that the computer implements the training method of any one of the image processing models or the image processing method.

[0190] In a possible implementation manner, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0191] In an example embodiment, a computer program product is also provided, and the computer program product includes a computer program or computer instructions. The computer program or computer instructions are loaded and executed by a processor, so that the computer implements the training method of any one of the image processing models or the image processing method.

[0192] It should be noted that the terms "first", "second" and the like in the description and in the claims are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the use of such terms as "first" and "second" are arbitrary labels. For example, a first element could be termed a second element and, similarly, a second element could be termed a first element without departing from the scope of the application. It is also to be understood that the use of the terms "first", "second", "third", etc. do not denote any order or sequence unless explicitly stated otherwise. The terms "top", "bottom", "front", "back", "side", "horizontal", "vertical", "upper", "lower", and the like are used to denote orientation in the figures and are not necessarily used according to the actual physical orientation of the apparatus or device in use. The embodiments of the application described herein are not meant to be limiting, but merely exemplary, as there are many embodiments of the application that are not specifically described herein. Thus, none of the aspects of the application should be deemed critical, essential, or required. In addition, it is intended that the methods and apparatus described herein can be used in any combination and are not limited to the combinations described herein.

[0193] It should be understood that "a plurality of" means two or more. "And / or" describes associated objects in association with the associated objects individually as well as in the aggregate. For example, "A and / or B" means only A, only B, or A and B.

[0194] The above description is intended to enable the person skilled in the art to carry out the application. Various alterations and modifications will be apparent to the person skilled in the art to which the application pertains, and it is intended that all such alterations and modifications be considered fair within the scope of the application.

Claims

1. A method for training an image processing model, characterized in that, The method comprises: acquiring a sample image with lighting effects and a label image corresponding to the sample image without lighting effects, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source being obtained by physical simulation on a real light source; training an initial image processing model using the sample image and the label image to obtain a target image processing model, the target image processing model being used for de-lighting processing on a to-be-processed image with lighting effects, the to-be-processed image being obtained by image acquisition on the reference object irradiated by a real light source; wherein the training of the initial image processing model using the sample image and the label image comprises: acquiring a first texture image corresponding to the sample image and a second texture image corresponding to the label image; inputting the first texture image into a generator in the initial image processing model for de-lighting processing to obtain a third texture image output by the generator; inputting the second texture image into a discriminator in the initial image processing model for discrimination to obtain a first discrimination result output by the discriminator, the first discrimination result being used for indicating a probability that the second texture image is an image output by the generator; inputting the third texture image into the discriminator for discrimination to obtain a second discrimination result output by the discriminator, the second discrimination result being used for indicating a probability that the third texture image is an image output by the generator; acquiring a first sub-loss function based on the first discrimination result and the second discrimination result; acquiring a first loss function based on the first sub-loss function; and training the initial image processing model using the first loss function.

2. The method of claim 1, wherein, The acquisition of the sample image with lighting effects and the label image corresponding to the sample image without lighting effects comprises: loading the virtual model of the reference object into a virtual light source environment, the virtual light source environment being provided with a virtual light source; performing image acquisition on the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment to obtain the sample image, and performing image acquisition on the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment to obtain the label image.

3. The method according to claim 1 or 2, characterized in that, The acquisition of the first loss function based on the first sub-loss function comprises: determining a first sub-image in the second texture image satisfying a reliable condition and a second sub-image in the third texture image satisfying the reliable condition; acquiring a second sub-loss function based on the first sub-image and the second sub-image; acquiring the first loss function based on the first sub-loss function and the second sub-loss function.

4. The method according to claim 1 or 2, characterized in that, The acquisition of the first texture image corresponding to the sample image comprises: determining a texture point corresponding to a virtual model point in a texture space and a mapping point corresponding to the virtual model point in the sample image, the virtual model point being a point constituting the virtual model of the reference object; determining a pixel value possessed by the texture point based on a pixel value possessed by the mapping point in the sample image; Based on the texture points and pixel values of the texture points, the first texture image is obtained.

5. The method according to claim 1 or 2, characterized in that, The training of the initial image processing model by using the sample image and the label image further includes: The sample image is input into the initial image processing model for de-illumination processing to obtain a predicted image output by the initial image processing model; Based on the predicted image and the label image, a second loss function is obtained, and the initial image processing model is trained by using the second loss function.

6. An image processing method characterized by, The method includes: obtaining a to-be-processed image with an illumination effect and a target image processing model, the target image processing model being trained by using the training method of the image processing model in any one of claims 1 to 5, the to-be-processed image being obtained by image acquisition on the reference object irradiated by a real light source; calling the target image processing model to perform de-illumination processing on the to-be-processed image to obtain a target image.

7. The method of claim 6, wherein, The calling of the target image processing model to perform de-illumination processing on the to-be-processed image to obtain a target image includes: obtaining a texture image corresponding to the to-be-processed image; inputting the texture image corresponding to the to-be-processed image into the target image processing model for de-illumination processing to obtain the target image output by the target image processing model.

8. The method of claim 6, wherein, The calling of the target image processing model to perform de-illumination processing on the to-be-processed image to obtain a target image includes: inputting the to-be-processed image into the target image processing model for de-illumination processing to obtain the target image output by the target image processing model.

9. A training device for an image processing model, characterized in that, The device includes: an obtaining unit configured to obtain a sample image with an illumination effect and a label image corresponding to the sample image without the illumination effect, the sample image being obtained by image acquisition on a virtual model of a reference object irradiated by a virtual light source, the virtual light source being obtained by physical simulation on a real light source; The training unit is configured to: acquire a first texture image corresponding to the sample image and a second texture image corresponding to the label image; input the first texture image into a generator in an initial image processing model for de-illumination processing to obtain a third texture image output by the generator; input the second texture image into a discriminator in the initial image processing model for discrimination to obtain a first discrimination result output by the discriminator, the first discrimination result being used to indicate a probability that the second texture image is an image output by the generator; input the third texture image into the discriminator for discrimination to obtain a second discrimination result output by the discriminator, the second discrimination result being used to indicate a probability that the third texture image is an image output by the generator; acquire a first sub-loss function based on the first discrimination result and the second discrimination result; acquire a first loss function based on the first sub-loss function; and train the initial image processing model by using the first loss function to obtain a target image processing model, the target image processing model being used for de-illumination processing of a to-be-processed image with an illumination effect, the to-be-processed image being obtained by image acquisition on the reference object irradiated by a real light source.

10. The apparatus of claim 9, wherein, The acquisition unit is configured to: load a virtual model of the reference object into a virtual light source environment, the virtual light source environment being provided with a virtual light source; and acquire an image of the virtual model of the reference object irradiated by the virtual light source in the virtual light source environment to obtain the sample image; and acquire an image of the virtual model of the reference object not irradiated by the virtual light source in the virtual light source environment to obtain the label image.

11. The apparatus of claim 9 or 10, wherein, The training unit is configured to: determine a first sub-image in the second texture image that meets a reliable condition and a second sub-image in the third texture image that meets the reliable condition; and acquire a second sub-loss function based on the first sub-image and the second sub-image. The first loss function is acquired based on the first sub-loss function and the second sub-loss function.

12. The apparatus of claim 9 or 10, wherein, The training unit is configured to: determine a texture point corresponding to a virtual model point in a texture space and a mapping point corresponding to the virtual model point in the sample image, the virtual model point being a point of the virtual model of the reference object; Determine a pixel value of the texture point based on a pixel value of the mapping point in the sample image. The training unit is configured to: input the sample image into the initial image processing model for de-illumination processing to obtain a predicted image output by the initial image processing model; acquire a second loss function based on the predicted image and the label image; and train the initial image processing model by using the second loss function.

13. The apparatus of claim 9 or 10, wherein, The device comprises:

14. An image processing apparatus characterized by comprising: ​ An acquisition unit is configured to acquire a to-be-processed image with lighting effects and a target image processing model, the target image processing model being trained by using the training method of the image processing model according to any one of claims 1 to 5, and the to-be-processed image being obtained by image acquisition on the reference object irradiated by a real light source. A processing unit is configured to call the target image processing model to perform de-lighting processing on the to-be-processed image to obtain a target image.

15. The apparatus of claim 14, wherein, The processing unit is configured to acquire a texture image corresponding to the to-be-processed image, input the texture image corresponding to the to-be-processed image into the target image processing model to perform de-lighting processing, and obtain the target image output by the target image processing model.

16. The apparatus of claim 14, wherein, The processing unit is configured to input the to-be-processed image into the target image processing model to perform de-lighting processing, and obtain the target image output by the target image processing model.

17. A computer device, comprising: The computer device includes a processor and a memory, and the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor, so that the computer device implements the training method of the image processing model according to any one of claims 1 to 5, or the image processing method according to any one of claims 6 to 8.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer implements the training method of the image processing model according to any one of claims 1 to 5, or the image processing method according to any one of claims 6 to 8.

19. A computer program product, characterised in that, The computer program product includes a computer program or computer instructions, and the computer program or the computer instructions are loaded and executed by the processor, so that the computer implements the training method of the image processing model according to any one of claims 1 to 5, or the image processing method according to any one of claims 6 to 8.

Images