Image processing method and device, equipment and storage medium

Abstract

The invention provides an image processing method and device, equipment and a storage medium, and relates to the field of image processing, in particular to the field of artificial intelligence. According to the specific implementation scheme, an original image is obtained, a derivative image is generated according to the original image, and the original image and the derivative image comprise the same target object; acquiring illumination information of the derivative image; and according to the illumination information, performing illumination adjustment processing on the target object in the derivative image to obtain a target derivative image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, in particular to the field of artificial intelligence, and specifically to an image processing method, apparatus, device and storage medium. Background Art

[0002] In modern digital image processing and generation technology, users often want to generate derivative images of the user (such as portrait images, stylized images) from a user image. However, this process faces a key challenge: the facial lighting of the user image is determined by the ambient light conditions when the image is taken. When the user's facial features are fused into the generated derivative image, if the generated ambient light conditions are inconsistent with the light conditions when the user was photographed, the facial lighting will be inconsistent, which will seriously affect the naturalness and authenticity of the generated image. Summary of the invention

[0003] The present disclosure provides an image processing method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, an image processing method is provided, which obtains an original image and generates a derivative image based on the original image, wherein the original image and the derivative image include the same target object; obtains lighting information of the derivative image; and performs lighting adjustment processing on the target object in the derivative image based on the lighting information to obtain a target derivative image.

[0005] The present application adjusts the brightness of the target object in the derived image according to the lighting information of the derived image, ensuring that the lighting conditions of the target object on the finally generated target derived image are consistent with those of the background, thereby avoiding lighting inconsistency.

[0006] According to another aspect of the present disclosure, there is provided an image processing device, including: an image generation module, used to acquire an original image, and generate a derivative image based on the original image, wherein the original image and the derivative image include the same target object; a first acquisition module, used to acquire lighting information of the derivative image; and a first adjustment module, used to perform lighting adjustment processing on the target object in the derivative image according to the lighting information, to obtain a target derivative image.

[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned image processing method.

[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned image processing method.

[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above-mentioned image processing method when executed by a processor.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of an exemplary implementation of an image processing method according to an exemplary embodiment of the present disclosure.

[0013] Figure 2 is a schematic diagram of an exemplary implementation of an image processing method according to an exemplary embodiment of the present disclosure.

[0014] Figure 3 is a schematic diagram of an exemplary implementation of an image processing method according to an exemplary embodiment of the present disclosure.

[0015] Figure 4 is a schematic diagram of an exemplary implementation of an image processing method according to an exemplary embodiment of the present disclosure.

[0016] Figure 5 is a schematic diagram of an exemplary implementation of an image processing method according to an exemplary embodiment of the present disclosure.

[0017] Figure 6 is an exemplary schematic diagram of an image processing device according to an exemplary embodiment of the present disclosure.

[0018] Figure 7 is a schematic block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Image processing is a technique that uses computers to analyze images to achieve the desired results. It is also called image processing. Image processing generally refers to digital image processing. A digital image is a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of the array are called pixels, and their values ​​are called grayscale values. Image processing technology generally includes three parts: image compression, enhancement and restoration, matching, description and recognition.

[0021] Artificial Intelligence (AI) is a discipline that studies how computers can simulate certain thought processes and intelligent behaviors of human beings (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technologies and software-level technologies. AI hardware technologies generally include computer vision technology, speech recognition technology, natural language processing technology, as well as learning / deep learning, big data processing technology, knowledge graph technology, and other aspects.

[0022] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0023] Figure 1 is a schematic diagram of an exemplary implementation of an image processing method shown in the present application, such as Figure 1 As shown, the image processing method comprises the following steps:

[0024] S101, acquiring an original image, and generating a derivative image based on the original image, wherein the original image and the derivative image include the same target object.

[0025] The original image generally refers to a real image obtained during the shooting process, and the target objects in the original image include but are not limited to people, scenery, vehicles, etc.

[0026] Among them, the derivative image refers to the derivative image generated according to the content of the original image through image processing and generation technology (such as deep learning, image enhancement, style transfer, etc.).

[0027] For example, if the original image is a portrait of a certain person, then a derivative image of the person can be generated based on the portrait. For example, the clothing, hairstyle, background, accessories, etc. in the generated derivative image may change, while the face of the person still uses the face of the person in the original image.

[0028] If the angle of the face of the person in the original image is different from the angle of the face of the person in the derived image, the angle of the face of the person in the derived image may be adaptively adjusted to better match the scene of the derived image.

[0029] S102, obtaining illumination information of the derived image.

[0030] Extract lighting information from the generated derivative image for subsequent lighting adjustment. Lighting information includes the direction, intensity, position, etc. of the light source.

[0031] Optionally, lighting information may be extracted from the derived image based on light and shadow decomposition technology.

[0032] S103, performing illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image.

[0033] The performing of illumination adjustment processing on the target object in the derived image mentioned here generally refers to performing of illumination adjustment processing on the target object or a local area of ​​the target object in the derived image.

[0034] Taking the target image as a person as an example, the area that needs to be adjusted for lighting is generally the exposed skin of the person, such as the face, neck, hands and other local areas.

[0035] Exemplarily, according to the lighting information, the brightness value of the part of the target object that needs to be subjected to lighting adjustment processing is enhanced or weakened to make it more consistent with the actual lighting environment.

[0036] The embodiment of the present application proposes an image processing method, which obtains an original image and generates a derivative image based on the original image, wherein the original image and the derivative image include the same target object; obtains lighting information of the derivative image; and performs lighting adjustment processing on the target object in the derivative image based on the lighting information to obtain a target derivative image. The present application adjusts the brightness of the target object in the derivative image based on the lighting information of the derivative image to ensure that the lighting conditions of the target object in the target derivative image finally generated are consistent with those of the background, thereby avoiding lighting inconsistency.

[0037] Figure 2 is a schematic diagram of an exemplary implementation of an image processing method shown in the present application, such as Figure 2 As shown, the image processing method comprises the following steps:

[0038] S201, obtaining an original image and derivative description information of the original image.

[0039] Among them, user images can be uploaded by users.

[0040] For example, the derived description information of the original image can be obtained by the image-to-text model. The derived description information usually includes the main elements, style features, colors, emotions, layout and other information in the original image.

[0041] For example, if the original image is a portrait photo, the derived description information may include: the person's gender, age, posture, facial expression, clothing, background style, lighting conditions, etc.

[0042] S202, generating a first prompt word according to the derived description information, and generating a derived image according to the first prompt word and the original image by a pre-trained text-based graph model, wherein the original image and the derived image include the same target object.

[0043] Keyword extraction is performed on the derived description information obtained above, and the extracted keywords are used as first prompt words, and the first prompt words are used to guide the text graph model to generate derived images.

[0044] The user may also directly input the first prompt word to subsequently guide the text graph model to generate a derivative image.

[0045] The first prompt word and the original image are input into the Wensheng graph model, which uses the information in the first prompt word to generate a derivative image that is related to the original image but different in style, details, etc.

[0046] Among them, the text-generated graph model can be pre-trained and generated based on a large number of original image samples and corresponding derived image samples.

[0047] S203, obtaining illumination information of the derived image.

[0048] S204: Determine a positional relationship between the light source and a target part of the target object according to the illumination information.

[0049] Based on the lighting information, determine the first position of the light source (such as the sun, lamps, etc.) in the derived image; obtain the second position of the target part of the target object (such as the face, hands) in the derived image; and determine the positional relationship between the light source and the target part of the target object based on the first position and the second position.

[0050] S205: Determine the front light area and the back light area of ​​the target part according to the position relationship.

[0051] The front light area refers to the area facing the light source, and the back light area refers to the area facing away from the light source.

[0052] For example, if the light source is located in the upper right corner and the person's face is located in the center of the image, it can be concluded that the right side of the person's face should be the face-lit area, and the left side of the person's face should be the backlit area, that is, there should be highlights on the right side of the person's face, and a slight shadow may appear on the left side of the person's face.

[0053] S206, adjusting the illumination of the front light area and the back light area to obtain a target derived image.

[0054] Determine the light intensity based on the lighting information.

[0055] The brightness enhancement value of the surface light area is determined according to the light intensity, and the brightness of the surface light area is enhanced according to the brightness enhancement value. The brightness enhancement value can be a fixed percentage of the entire surface light area, an adjustment value of the entire surface light area, or an adjustment value of each pixel in the entire surface light area, which changes according to the change of light intensity.

[0056] The brightness reduction value of the backlight area is determined according to the light intensity, and the brightness of the backlight area is reduced according to the brightness reduction value to obtain a target derived image. The brightness reduction value can be a fixed percentage of the entire backlight area, an adjustment value of the entire backlight area, or an adjustment value of each pixel in the entire backlight area, which changes according to the change of light intensity.

[0057] The embodiment of the present application obtains derivative description information of the original image and generates a first prompt word based on this information, so that the text image model can understand the original image more accurately, thereby generating a derivative image with artistic value; by analyzing the lighting information of the derivative image, the positional relationship between the light source and the target object is determined, and then precise lighting adjustments are made to the facelight area and the backlight area, so that the generated image has a more natural and realistic lighting effect.

[0058] Figure 3 is a schematic diagram of an exemplary implementation of an image processing method shown in the present application, such as Figure 3 As shown, the image processing method comprises the following steps:

[0059] S301, obtaining an original image and derivative description information of the original image.

[0060] S302, generating a first prompt word according to the derived description information, and generating a derived image according to the first prompt word and the original image by a pre-trained text-based graph model, wherein the original image and the derived image include the same target object.

[0061] S303, obtaining illumination information of the derived image.

[0062] Regarding the specific implementation of steps S301 to S303, reference may be made to the specific introduction of the relevant parts in the above embodiments, which will not be elaborated here.

[0063] S304: extracting a mask image of a target part of the target object from the original image.

[0064] For example, if the target object is a person and the target part is the face of the person, the facial region of the person is segmented from the original image, and then a mask image of the facial region is generated.

[0065] Furthermore, after extracting the mask image, the edge of the mask image can be feathered so that when the target derivative image is subsequently generated, the transition between the target part and the background is smoother and more natural, avoiding the abruptness caused by the hard boundary.

[0066] S305: Perform light and shadow migration processing on the mask image of the target part according to the illumination information, and update it into the derived image.

[0067] According to the illumination information, the light and shadow migration technology is used to migrate the illumination information to the mask map of the target part, and the processed mask map is fused with the derived image.

[0068] S306 , extracting the edge region of the mask image of the target part, and redrawing the edge region of the target part in the derived image based on the diffusion model to obtain a target derived image.

[0069] The erosion-dilation technique is used to obtain the edge area of ​​the mask image of the target part, and the diffusion model is used to redraw the edge area to improve the naturalness of the edge.

[0070] The embodiment of the present application obtains derivative description information of the original image and generates a first prompt word based on this information, so that the Wensheng graph model can understand the original image more accurately, thereby generating a derivative image with artistic value; according to the lighting information in the derivative image, the light and shadow migration technology is used to process the mask image of the target part, so that the processed target part can be consistent with the lighting and shadow information in the derivative image, so that the target part is more natural and realistic in the target derivative image; using the diffusion model to redraw the edge area of ​​the target part helps to optimize the details and naturalness of the edge and reduce the abruptness of the hard boundary.

[0071] Figure 4 is a schematic diagram of an exemplary implementation of an image processing method shown in the present application, such as Figure 4 As shown, the image processing method comprises the following steps:

[0072] S401, acquiring an original image, and generating a derivative image according to the original image, wherein the original image and the derivative image include the same target object.

[0073] Regarding the specific implementation of step S401, please refer to the specific introduction of the relevant parts in the above embodiment, which will not be repeated here.

[0074] S402: Acquire morphological features of a designated part of a target object in an original image.

[0075] The original image is input into the pre-trained target processing model, and the target processing model extracts the morphological features of the designated part and outputs the morphological features of the designated part. Wherein, when the target object is a person, the designated part of the target object includes at least one of the face and the hand of the person.

[0076] When the designated part of the target object is a person's face, the target processing model is a face processing model.

[0077] When the designated part of the target object is a person's hand, the target processing model is a hand processing model.

[0078] Among them, when the designated parts of the target object are the face and hands of a person, the target processing model is a face processing model and a hand processing model.

[0079] Among them, if there are multiple designated parts (for example, the above-mentioned case includes both hands and face), if the skin color of one of the designated parts is monitored and adjusted, the skin color of other designated parts is synchronously adjusted according to the adjusted skin color of one of the designated parts, thereby ensuring the consistency of skin color in the entire image.

[0080] Among them, for the target processing model corresponding to any designated part, the training process of the target processing model includes: obtaining a standard portrait derivative image and obtaining multimodal description information of the portrait derivative image (for example, the multimodal model can be used to obtain visual information, text information, etc. of the portrait derivative image); segmenting the designated part of the face derivative image to obtain the designated part image; performing illumination decomposition and structure retention on the designated part image to obtain the marked morphological features of the designated part; based on the multimodal description information, generating a second prompt word, and optimizing the base model according to the portrait derivative image, the second prompt word and the marked morphological features to obtain the target processing model. The target processing model trained in this way can more accurately extract the morphological features of the designated part, thereby performing morphological adjustments on the designated part of the target object in the derivative image, and preventing the final generated target derivative image from having hand and face distortion.

[0081] The morphological features include at least texture structure information and contour information of the designated part.

[0082] The contour information refers to the boundary line or edge of a specified part of the target object.

[0083] S403: Perform morphological adjustment on the designated part of the target object in the derived image according to the morphological features of the designated part.

[0084] S404, obtaining illumination information of the derived image.

[0085] S405 , performing illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image.

[0086] Regarding the specific implementation method of steps S404 to S405, please refer to the specific introduction of the relevant parts in the above embodiments. The embodiment of the present application only arranges these two steps after the morphological adjustment of the designated parts of the target object in the derived image. The specific implementation method of the steps has not changed and will not be repeated here.

[0087] The embodiment of the present application can significantly improve the quality of the final target derived image through morphological adjustment, make the texture of the specified part more natural and delicate, and prevent the final generated target derived image from having hand and face distortion.

[0088] It is not difficult to understand that the above steps S402 to S403 are to perform morphological adjustment on the designated part of the target object in the derived image. Figure 4 The embodiment shown arranges the morphological adjustment of the designated part of the target object in the derived image before the illumination adjustment processing of the target object in the derived image. In practical applications, the morphological adjustment of the designated part of the target object in the derived image can also be arranged after the illumination adjustment processing of the target object in the derived image. Based on this idea, the present application proposes Figure 5 , Figure 5 is a schematic diagram of an exemplary implementation of an image processing method shown in the present application, such as Figure 5 As shown, the image processing method comprises the following steps:

[0089] S501, acquiring an original image, and generating a derivative image according to the original image, wherein the original image and the derivative image include the same target object.

[0090] S502, obtaining illumination information of the derived image.

[0091] S503: Performing illumination adjustment processing on the target object in the derived image according to the illumination information.

[0092] Regarding the specific implementation of steps S501 to S503, reference may be made to the specific introduction of the relevant parts in the above embodiments, which will not be elaborated here.

[0093] S504: Acquire the morphological features of the designated part of the target object in the original image.

[0094] S505 , performing morphological adjustment on the designated part of the target object in the derived image after the illumination adjustment process according to the morphological features of the designated part, to obtain a target derived image.

[0095] Regarding the specific implementation method of steps S504 to S505, please refer to the specific introduction of the relevant parts in the above embodiments. The embodiment of the present application only arranges these two steps after the lighting adjustment processing of the target object in the derived image, and the specific implementation method of the steps has not changed, so it will not be repeated here.

[0096] The embodiment of the present application can significantly improve the quality of the final target derived image through morphological adjustment, make the texture of the specified part more natural and delicate, and prevent the final generated target derived image from having hand and face distortion.

[0097] Figure 6 is an exemplary schematic diagram of an image processing device shown in the present application, such as Figure 6 As shown, the image processing device 600 includes an image generating module 601, a first acquiring module 602 and a first adjusting module 603, wherein:

[0098] The image generation module 601 is used to obtain an original image and generate a derivative image based on the original image. The original image and the derivative image include the same target object.

[0099] The first acquisition module 602 is used to acquire illumination information of the derived image.

[0100] The first adjustment module 603 is used to perform illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image.

[0101] The device adjusts the illumination information of the target object in the derived image according to the illumination information of the derived image, ensuring that the illumination of the target object on the finally generated target derived image is consistent with the illumination condition of the background, thereby avoiding the occurrence of illumination incoordination.

[0102] Furthermore, the image generation module 601 is also used to: obtain derived description information of the original image; generate a first prompt word according to the derived description information, and generate a derived image according to the first prompt word and the original image by a pre-trained text-based graph model.

[0103] Furthermore, the first adjustment module 603 is also used to: determine the positional relationship between the light source and the target part of the target object according to the lighting information; determine the surface light area and the backlight area of ​​the target part according to the positional relationship; and perform lighting adjustment on the surface light area and the backlight area to obtain a target derived image.

[0104] Furthermore, the first adjustment module 603 is also used to: determine a first position of the light source in the derived image based on the lighting information; obtain a second position of the target part of the target object in the derived image; and determine a positional relationship between the light source and the target part of the target object based on the first position and the second position.

[0105] Furthermore, the first adjustment module 603 is also used to: determine the light intensity according to the lighting information; determine the brightness enhancement value of the surface light area according to the light intensity, and enhance the brightness of the surface light area according to the brightness enhancement value; determine the brightness reduction value of the backlight area according to the light intensity, and reduce the brightness of the backlight area according to the brightness reduction value to obtain a target derived image.

[0106] Furthermore, the first adjustment module 603 is also used to: extract a mask map of a target part of the target object from the original image; perform light and shadow migration processing on the mask map of the target part according to the lighting information, and update it to the derived image; extract the edge area of ​​the mask map of the target part, and redraw the edge area of ​​the target part in the derived image based on the diffusion model to obtain the target derivative image.

[0107] Furthermore, the image processing device 600 also includes: a second acquisition module, used to acquire the morphological features of the specified part of the target object in the original image; and a second adjustment module, used to perform morphological adjustments on the specified part of the target object in the derived image according to the morphological features of the specified part.

[0108] Furthermore, when the target object is a person, the designated part of the target object includes at least one of the face and the hand of the person.

[0109] Furthermore, the second adjustment module is also used for: in response to there being multiple designated parts, in response to monitoring the skin color of one of the designated parts, making adjustments, and synchronously adjusting the skin color of another designated part according to the adjusted skin color of one of the designated parts.

[0110] Furthermore, the second acquisition module is also used to: input the original image into a pre-trained target processing model, and the target processing model extracts morphological features of the specified part, and outputs the morphological features of the specified part; wherein the target processing model includes at least one of a hand processing model and a face processing model.

[0111] Furthermore, the image processing device 600 also includes a model training module, which is used to: obtain a standard portrait-derived image and obtain multimodal description information of the portrait-derived image; segment a specified part of the face-derived image to obtain an image of the specified part; perform illumination decomposition and structure retention on the image of the specified part to obtain marked morphological features of the specified part; generate a second prompt word based on the multimodal description information, and optimize and train the base model according to the portrait-derived image, the second prompt word and the marked morphological features to obtain a target processing model.

[0112] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0113] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0114] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0115] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0116] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

[0117] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0119] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0122] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, wherein: The method comprises: Acquire an original image, and generate a derivative image based on the original image, wherein the original image and the derivative image include the same target object; Acquiring illumination information of the derived image; According to the illumination information, illumination adjustment processing is performed on the target object in the derived image to obtain a target derived image.

2. The method according to claim 1, wherein: Generating a derivative image according to the original image includes: Obtaining derivative description information of the original image; A first prompt word is generated according to the derived description information, and a pre-trained text-based graph model is used to generate the derived image according to the first prompt word and the original image.

3. The method according to claim 1 or 2, wherein: The step of performing illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image includes: Determining a positional relationship between a light source and a target part of the target object according to the illumination information; Determine the surface light area and the backlight area of ​​the target part according to the positional relationship; The illumination of the surface light area and the backlight area is adjusted to obtain the target derived image.

4. The method according to claim 3, wherein: The determining, according to the illumination information, a positional relationship between a light source and a target part of the target object comprises: Determining a first position of the light source in the derived image according to the illumination information; Acquire a second position of a target part of the target object in the derived image; A positional relationship between the light source and a target portion of the target object is determined according to the first position and the second position.

5. The method according to claim 4, wherein: The step of adjusting the illumination of the surface light area and the backlight area to obtain the target derived image includes: Determining light intensity according to the lighting information; Determining a brightness enhancement value of the surface light area according to the light intensity, and performing brightness enhancement on the surface light area according to the brightness enhancement value; A brightness reduction value of the backlight area is determined according to the light intensity, and the brightness of the backlight area is reduced according to the brightness reduction value to obtain the target derived image.

6. The method according to claim 2, wherein: The step of performing illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image includes: Extracting a mask image of a target part of the target object from the original image; According to the illumination information, performing light and shadow migration processing on the mask image of the target part, and updating the mask image into the derived image; The edge region of the mask image of the target part is extracted, and the edge region of the target part in the derived image is redrawn based on a diffusion model to obtain the target derived image.

7. The method according to claim 1, wherein: The method further comprises: Acquire the morphological features of the designated part of the target object in the original image; The designated part of the target object in the derived image is morphologically adjusted according to the morphological features of the designated part.

8. The method according to claim 7, wherein: When the target object is a person, the designated part of the target object includes at least one of the face and the hand of the person.

9. The method according to claim 7 or 8, wherein: The method further comprises: In response to the number of designated parts being multiple, in response to the skin color of one designated part being monitored and adjusted, the skin colors of other designated parts are synchronously adjusted according to the adjusted skin color of the one designated part.

10. The method according to claim 9, wherein: The acquiring the morphological features of the designated part of the target object in the original image includes: Inputting the original image into a pre-trained target processing model, the target processing model extracts morphological features of the designated part, and outputs the morphological features of the designated part; Wherein, the target processing model includes at least one of a hand processing model and a face processing model.

11. The method according to claim 10, wherein: The training process of the target processing model includes: Acquire a standard portrait-derived image, and acquire multimodal description information of the portrait-derived image; Segmenting the designated part of the face-derived image to obtain an image of the designated part; Performing illumination decomposition and structure preservation on the designated part image to obtain the marked morphological features of the designated part; Based on the multimodal description information, a second prompt word is generated, and according to the portrait-derived image, the second prompt word and the mark morphological features, the base model is optimized and trained to obtain the target processing model.

12. An image processing device, wherein: The device comprises: An image generation module, used for acquiring an original image and generating a derivative image according to the original image, wherein the original image and the derivative image include the same target object; A first acquisition module, used for acquiring illumination information of the derived image; The first adjustment module is used to perform illumination adjustment processing on the target object in the derived image according to the illumination information to obtain a target derived image.

13. The device according to claim 12, wherein: The image generation module is further used for: Obtaining derivative description information of the original image; A first prompt word is generated according to the derived description information, and a pre-trained text-based graph model is used to generate the derived image according to the first prompt word and the original image.

14. The device according to claim 12 or 13, wherein: The first adjustment module is further used for: Determining a positional relationship between a light source and a target part of the target object according to the illumination information; Determine the surface light area and the backlight area of ​​the target part according to the positional relationship; The illumination of the surface light area and the backlight area is adjusted to obtain the target derived image.

15. The device according to claim 14, wherein: The first adjustment module is further used for: Determining a first position of the light source in the derived image according to the illumination information; Acquire a second position of a target part of the target object in the derived image; A positional relationship between the light source and a target portion of the target object is determined according to the first position and the second position.

16. The device according to claim 15, wherein: The first adjustment module is further used for: Determining light intensity according to the lighting information; Determining a brightness enhancement value of the surface light area according to the light intensity, and performing brightness enhancement on the surface light area according to the brightness enhancement value; A brightness reduction value of the backlight area is determined according to the light intensity, and the brightness of the backlight area is reduced according to the brightness reduction value to obtain the target derived image.

17. The device according to claim 13, wherein: The first adjustment module is further used for: Extracting a mask image of a target part of the target object from the original image; According to the illumination information, performing light and shadow migration processing on the mask image of the target part, and updating the mask image into the derived image; The edge region of the mask image of the target part is extracted, and the edge region of the target part in the derived image is redrawn based on a diffusion model to obtain the target derived image.

18. The device according to claim 12, wherein: The device further comprises: A second acquisition module is used to acquire the morphological features of the designated part of the target object in the original image; The second adjustment module is used to perform morphological adjustment on the designated part of the target object in the derived image according to the morphological features of the designated part.

19. The device according to claim 18, wherein: When the target object is a person, the designated part of the target object includes at least one of the face and the hand of the person.

20. The device according to claim 18 or 19, wherein: The second adjustment module is further used for: In response to the number of designated parts being multiple, in response to the skin color of one designated part being monitored and adjusted, the skin colors of other designated parts are synchronously adjusted according to the adjusted skin color of the one designated part.

21. The device according to claim 20, wherein: The second acquisition module is further used for: Inputting the original image into a pre-trained target processing model, the target processing model extracts morphological features of the designated part, and outputs the morphological features of the designated part; Wherein, the target processing model includes at least one of a hand processing model and a face processing model.

22. The device according to claim 21, wherein The device also includes a model training module, and the model training module is used to: Acquire a standard portrait-derived image, and acquire multimodal description information of the portrait-derived image; Segmenting the designated part of the face-derived image to obtain an image of the designated part; Performing illumination decomposition and structure preservation on the designated part image to obtain the marked morphological features of the designated part; Based on the multimodal description information, a second prompt word is generated, and according to the portrait-derived image, the second prompt word and the mark morphological features, the base model is optimized and trained to obtain the target processing model.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

25. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images