Image processing method and apparatus, device, and medium

Abstract

Embodiments of the present disclosure relate to an image processing method and apparatus, a device, and a medium. The method comprises: acquiring a first image to be processed; detecting a target area in the first image, and masking the detected target area to obtain a second image; retouching on the basis of the second image to obtain a target image, wherein a target area in the target image is different from the target area in the first image.

Description

Image processing method, device, equipment and medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese invention patent application number 202310762692.2, entitled “Image processing method, device, equipment and medium” and filed on June 26, 2023, and the entire application is incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and medium. Background Art

[0004] Image editing functions are widely used in various applications, such as image editing software, photo taking software, and live video streaming platforms. Users can adjust images as needed, such as applying beautification to facial images. However, the inventors have discovered that existing image editing functions mainly automatically edit the entire face or body, or require users to spend time and effort manually editing the image, which makes it difficult for some users to easily adjust local areas of an image.

[0005] Summary of the Invention

[0006] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an image processing method, apparatus, device and medium.

[0007] An embodiment of the present disclosure provides an image processing method, which includes: acquiring a first image to be processed; detecting a target area in the first image, and performing occlusion processing on the detected target area to obtain a second image; performing repair processing based on the second image to obtain a target image; wherein the target area in the target image is different from the target area in the first image.

[0008] Optionally, the repair processing based on the second image to obtain the target image includes: detecting a specified sub-region in the target area in the first image to obtain a sub-region detection result; and performing repair processing based on the sub-region detection result and the second image to obtain the target image.

[0009] Optionally, the designated sub-region in the target image matches the designated sub-region in the first image.

[0010] Optionally, the sub-region detection result is presented in the form of a lower teeth region mask image; the repair processing based on the sub-region detection result and the second image includes: inputting the specified sub-region mask image and the second image into a preset neural network model, so that the neural network model performs repair processing on the target area in the second image based on the sub-region detection result.

[0011] Optionally, the neural network model is obtained according to the following steps: obtaining a first sample image; detecting the target area in the first sample image, and performing occlusion processing on the detected target area to obtain a second sample image; obtaining a specified sub-area mask image corresponding to the first sample image, and inputting the second sample image and the specified sub-area mask image into the initial neural network model to obtain a generated image output by the initial neural network model; adjusting the parameters of the initial neural network model based on the generated image, the first sample image and a preset loss function until a neural network model that meets the preset conditions is obtained.

[0012] Optionally, the aesthetic level of the target area in the first sample image is greater than a preset threshold.

[0013] Optionally, adjusting the parameters of the initial network model based on the generated image, the first sample image and a preset loss function includes: obtaining an adversarial loss value between the generated image and the first sample image based on a preset adversarial loss function; obtaining a pixel loss value between the generated image and the first sample image based on a preset pixel loss function; and adjusting the parameters of the initial network model according to the adversarial loss value and the pixel loss value.

[0014] Optionally, the detected target area is occluded to obtain a second image, including: generating a target area mask map based on the detected target area; generating a second image according to the target area mask map and the first image; wherein the pixel values ​​of the target area in the second image are all specified pixel values.

[0015] An embodiment of the present disclosure also provides an image processing device, including: an image acquisition module, used to acquire a first image to be processed; a region occlusion module, used to detect a target region in the first image and perform occlusion processing on the detected target region to obtain a second image; an image repair module, used to perform repair processing on the target image based on the second image to obtain a target image; wherein, there is a difference between the target region in the target image and the target region in the first image.

[0016] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the image processing method provided by the embodiment of the present disclosure.

[0017] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the image processing method provided by the embodiment of the present disclosure.

[0018] The technical solution provided by the embodiments of the present disclosure can detect a target area in a first image to be processed, perform occlusion processing on the detected target area to obtain a second image, and further perform restoration processing based on the second image to obtain a target image, in which the target area in the target image differs from the target area in the first image.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] FIG1 is a schematic flow chart of an image processing method provided by an embodiment of the present disclosure;

[0023] FIG2 is a schematic diagram of a model acquisition provided by an embodiment of the present disclosure;

[0024] FIG3 is a flow chart of an image processing method provided by an embodiment of the present disclosure;

[0025] FIG4 is a comparison diagram of effects of an image processing method provided by an embodiment of the present disclosure;

[0026] FIG5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure;

[0027] FIG6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] FIG1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. The method can be performed by an image processing device, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device. As shown in FIG1 , the method mainly includes the following steps S102 to S106:

[0031] Step S102: Acquire a first image to be processed. Exemplarily, the first image is an image containing a target area. The target area may be, for example, a local area of ​​a target object. The target object may be, for example, a person, a cat, or a specific object. The local area of ​​the target object may be, for example, a body part of a person or animal, such as teeth, eyes, or arms. It may also be a wheel of a vehicle, a trunk of a tree, or the like. The disclosed embodiment does not restrict the target object and the local area of ​​the target object, and they may be flexibly configured as needed. In addition, the disclosed embodiment does not restrict the method for acquiring the first image. For example, it may be an image uploaded by a user, an image selected locally by a user, an image downloaded by a user via the Internet, or an image transmitted via other devices.

[0032] Step S104 : detecting the target area in the first image and performing occlusion processing on the detected target area to obtain a second image.

[0033] In practical applications, the target area can be detected using an existing target detection model. The detection principle includes, but is not limited to, determining the target area by detecting multiple key points. Taking the mouth area of ​​the target object as an example, the mouth area can be determined by detecting multiple mouth key points. To facilitate processing of the target area, the embodiment of the present disclosure directly performs occlusion processing (also known as masking processing) on ​​the target area. The embodiment of the present disclosure does not limit the occlusion processing method. For example, the pixel values ​​in the target area can be uniformly changed to preset special pixel values, such as setting the RGB pixel values ​​to (0, 0, 0), that is, the target area is rendered entirely black.

[0034] Step S106 , performing restoration processing based on the second image to obtain a target image; wherein the target area in the target image is different from the target area in the first image.

[0035] The embodiments of the present disclosure do not limit the method of restoration processing. For example, the occluded target area in the second image can be filled based on image restoration technology. For example, if the target area is the mouth area, the teeth of the occluded mouth area in the second image can be filled based on image restoration technology. In specific implementation, the neural network model provided by this application (also known as the region adjustment model) can be used to directly process the second image. The target area in the obtained target image is generated by the neural network model and has certain differences from the original target area of ​​the target object in the first image. For example, the neural network model can generate a more beautiful target area, thereby achieving beautification of the target area. That is, the beauty of the target area in the target image is higher than that of the target area in the first image. For example, if the target area is the mouth area of ​​the target object, the neural network model can be used to fill and generate a more beautiful mouth area. Specifically, the neural network model is capable of filling and generating beautiful teeth. That is, the beauty of the teeth of the target object in the target image is higher than that of the teeth of the target object in the first image, thereby making the mouth of the target object have a better overall visual appearance.

[0036] That is, the target area in the first image can be optimized and adjusted through the above method. Of course, the optimization and adjustment of the target area is only an example. In actual applications, the target area can also be adjusted as needed, such as adjusting it to a target area with a specific look and feel. For ease of understanding, taking the target area as the mouth as an example, it is not limited to optimizing and adjusting the teeth in the mouth. The teeth in the mouth can also be adjusted to special effect fangs, etc. There is no limitation here. It mainly depends on the characteristics of the neural network model or the model training method, and the specific setting can be flexible.

[0037] By using the above method of first blocking and then repairing the target area, the target area in the image can be adjusted conveniently and effectively, which can better meet user needs.

[0038] The embodiment of the present disclosure first provides a method for performing occlusion processing on the detected target area to obtain a second image, which can be performed with reference to the following steps (1) and (2):

[0039] Step (1) generates a target area mask based on the detected target area. In practical applications, the detected target area can be segmented to generate a target area mask. The target area mask can be a binary image composed of pixel values ​​0 and 1. The target area mask can be used to clearly identify the target area and the non-target area in the first image. The pixel values ​​of the target area and the non-target area are different.

[0040] Step (2) generates a second image based on the target area mask and the first image; wherein the pixel values ​​of the target area in the second image are all specified pixel values, and the pixel values ​​of the non-target area in the second image are consistent with the pixel values ​​of the non-target area in the first image. In practical applications, the target area and non-target area in the first image can be clearly determined based on the target area mask, and the pixel values ​​of the target area in the second image are uniformly set to specified pixel values, such as the RGB pixel values ​​(0, 0, 0) corresponding to black, while the pixel values ​​of the non-target area in the second image are still equal to the pixel values ​​of the non-target area in the first image, that is, the pixel values ​​of the non-target area are kept unchanged. In this way, a second image with the target area blocked can be obtained efficiently and reliably.

[0041] In order to more accurately perform the restoration process and obtain a target image that better meets user needs, the present embodiment provides a specific implementation example of performing the restoration process based on the second image to obtain the target image, which can be performed with reference to the following steps A and B:

[0042] Step A, detects the designated sub-region in the target area in the first image to obtain the sub-region detection result. The designated sub-region can be flexibly set according to the needs, usually depending on the type of the target area, such as, if the target area is the mouth area, the designated sub-region can be the lower teeth area in the mouth area; for example, if the target area is the eye area, the designated sub-region can be the pupil area; for example, if the target area is the tree trunk area, the designated sub-region can be the tree hole area. In practical applications, the target detection algorithm can be used to detect the designated sub-region. Specifically, the corresponding detection algorithm can be used for detection according to the type of the designated sub-region. For example, if the designated sub-region is the lower teeth area, the lower teeth detection algorithm is used to detect the lower teeth area. For example, the lower teeth area can be determined by detecting the lower teeth key points, or the lower teeth area can be detected and segmented using a lower teeth segmentation model.

[0043] It should be noted that a specified sub-region may or may not exist in the target region, so the sub-region detection result will indicate whether the specified sub-region exists in the target region. For ease of understanding, taking the target object as a person and the target region as the mouth region of the person as an example, when a person opens his or her mouth, he or she may show both upper and lower teeth, or only upper teeth without showing lower teeth; therefore, the lower teeth detection result obtained by detecting the lower teeth region of the target object in the first image may indicate that the first image shows the lower teeth of the target object, or may indicate that the first image does not show the lower teeth of the target object, that is, the lower teeth of the target object are not detected.

[0044] Step B: performing restoration processing based on the sub-region detection result and the second image to obtain a target image.

[0045] The sub-region detection results can be used to constrain the restoration process of the second image, avoiding the situation where a specified sub-region originally existed in the first image but was not generated during the restoration process, or avoiding the situation where a specified sub-region originally did not exist in the first image but was additionally generated during the restoration process. By pre-detecting the specified sub-region in the first image, the sub-region detection results are referenced during the restoration process, and the specified sub-region in the target image is matched with the specified sub-region in the first image. For example, the meaning of matching can indicate that the specified sub-regions of the two images are consistent or the difference is less than a preset difference threshold. When the first image has a specified sub-region, the target image also has a specified sub-region. When the first image does not have a specified sub-region, the target image also has a specified sub-region. In this way, the structure of the target region can be matched with the structure of the target region of the first image, effectively ensuring the target region matching degree between the target image and the first image, and ensuring the user's visual experience. For example, taking the designated sub-region as the lower teeth region as an example, it can effectively ensure that the tooth structure of the target image matches the tooth structure of the first image, avoiding the situation where there are lower teeth in the first image but the lower teeth are not generated during the restoration process, or avoiding the situation where there are no lower teeth in the first image but additional lower teeth are generated during the restoration process, thereby ensuring a high degree of tooth matching before and after image restoration.

[0046] In some embodiments, the sub-region detection result is presented in the form of a designated sub-region mask. On this basis, in the specific implementation of step B, the designated sub-region mask and the second image can be input into a preset neural network model, so that the neural network model performs repair processing on the target region in the second image based on the sub-region detection result. The disclosed embodiment does not limit the structure of the neural network model. For example, the neural network model can include a generative adversarial network, specifically a generator. In actual applications, the neural network model can determine the designated sub-region in the occluded target region in the second image based on the designated sub-region mask, and repair the occluded target region in the second image. Specifically, the target region can be filled based on the content type of the target region and the content type of the designated sub-region, and the designated sub-region in the filling result matches the sub-region detection result. For example, if the target region is the mouth region and the designated sub-region is the lower teeth region, the upper teeth in the occluded mouth region can be generated. When the designated sub-region mask indicates the presence of lower teeth, the neural network model will also generate the lower teeth in the occluded mouth region, thereby obtaining the target image.

[0047] The present disclosure also provides a method for obtaining the above-mentioned preset neural network model. Specifically, the following steps a to d may be referred to for execution:

[0048] Step a: Acquire a first sample image.

[0049] The first sample image is an image containing a target area. For example, if the target area is the mouth area of ​​a target object, the target object in the first sample image can be an object with an open mouth, and the teeth of the target object can be clearly exposed. It should be noted that the target object in the first sample image and the target object in the first image can be considered to belong to the same type of object, such as both are humans, and should not be considered to belong to the same individual, that is, they are not necessarily images of the same person. In actual applications, the number of first sample images is usually multiple, such as images of multiple people's smiling faces with their teeth exposed.

[0050] In some implementation examples, the aesthetic level of the target region in the first sample image is greater than a preset threshold. In other words, the first sample image is a sample image with a relatively high aesthetic level for the target region, so that a neural network model trained using the first sample image can also generate an image with an aesthetically pleasing target region, thereby achieving optimized adjustment of the target region.

[0051] Step b: Detect the target area in the first sample image and perform occlusion processing on the detected target area to obtain a second sample image. The target area detection method and the target area occlusion method can be specifically referred to the above related content and will not be repeated here.

[0052] Step c, obtain the designated sub-region mask map corresponding to the first sample image, and input the second sample image and the designated sub-region mask map corresponding to the first sample image into the initial neural network model to obtain the generated image output by the initial neural network model. The method for obtaining the designated sub-region mask map corresponding to the first sample image can refer to the aforementioned related content and will not be repeated here. It should be noted that the initial neural network model mentioned here has the same structure as the preset neural network model used for processing the second image before, but has different network parameters. By continuously adjusting the parameters of the initial neural network model, the aforementioned preset neural network model can be finally obtained. The initial neural network model can generate a designated sub-region in the occluded target area in the second sample image under the constraint of the designated sub-region mask map corresponding to the first sample image, thereby obtaining a generated image.

[0053] Step d: Adjust the parameters of the initial neural network model based on the generated image, the first sample image and the preset loss function until a neural network model that meets the preset conditions is obtained.

[0054] It is understandable that if the parameters of the neural network model are not good, the output generated image does not meet the requirements, so it is necessary to further adjust the parameters of the neural network model to optimize the target area generation effect of the neural network model. Specifically, the preset loss function can be used to measure the difference between the generated image and the first sample image, so as to reversely adjust the parameters of the initial neural network model with the goal of reducing the difference until the preset conditions are met. The preset conditions include but are not limited to the difference between the generated image and the first sample image (which can be represented by a target loss value) being less than a preset threshold or reaching a preset convergence condition. Under normal circumstances, a neural network model that meets the preset conditions can output a generated image that meets the requirements.

[0055] Exemplarily, the preset loss function includes an adversarial loss function and / or a pixel loss function. In some specific implementation examples, step d can be performed with reference to the following steps d1 to d3:

[0056] Step d1, based on a preset adversarial loss function, obtains the adversarial loss value between the generated image and the first sample image. In practical applications, the generated image and the first sample image can be input into a discriminator, and the discriminator outputs the adversarial loss value between the generated image and the first sample image to measure the authenticity of the generated image. The discriminator is mainly used to determine whether the generated image and the first sample image are real or fake images. For example, if the discriminator can clearly identify that the generated image is a fake image and the first sample image is a real image, it means that the authenticity of the generated image is poor, and the adversarial loss value is also high. If the discriminator believes that the generated image is a real image, the authenticity of the generated image is high. In other words, the generated image output by the neural network model is more realistic, so realistic that the discriminator can confuse the generated image as a real image, and the adversarial loss value is low.

[0057] In step d2, a pixel loss value between the generated image and the first sample image is obtained based on a preset pixel loss function. The pixel loss function is used to measure the pixel difference between the generated image and the first sample image. The smaller the pixel difference, the more similar the generated image is to the first sample image.

[0058] Step d3: Adjust the parameters of the initial neural network model according to the adversarial loss value and the pixel loss value.

[0059] In practical applications, the weight corresponding to the adversarial loss value and the weight corresponding to the pixel loss value can be obtained. Based on the weights corresponding to the two, the adversarial loss value and the pixel loss value are weighted to obtain the target loss value (that is, the total loss value), thereby adjusting the parameters of the initial network model based on the target loss value. The parameters of the initial neural network model are adjusted in the direction of reducing the target loss value until the target loss value converges to less than the preset threshold. At this time, the neural network model can already output a generated image that is similar to the first sample image and has a strong sense of reality. If the target area of ​​the target object in the first sample image has a high degree of aesthetics, the neural network model can also generate an image containing a more aesthetically pleasing target area, which can be applied to subsequent target area beautification scenes.

[0060] For ease of understanding, taking the target area as the mouth area and the designated sub-area in the target area as the lower teeth area as an example, a model acquisition schematic diagram can be referred to as shown in FIG2 , which illustrates that the first sample image is segmented into the mouth area and the lower teeth area respectively, thereby obtaining a mouth area mask map (corresponding to the aforementioned target area mask map) and a lower teeth area mask map (corresponding to the aforementioned designated sub-area mask map), and further performing mouth area occlusion processing based on the mouth area mask map to generate a second sample image. Afterwards, the lower teeth area mask map and the second sample image can be input into the generator, and the generator outputs the generated image. The subsequent output image and the first sample image are simultaneously input into the discriminator to obtain an adversarial loss value through the discriminator, and a pixel loss value is obtained based on the generated image and the first sample image. Subsequently, a total loss value can be obtained based on the weighted adversarial loss value and the pixel loss value, and the parameters of the generator can be adjusted based on the total loss value until a generator whose total loss value meets the preset conditions is obtained. The generator can be used as the aforementioned preset neural network model that can directly perform repair processing on the mouth area. The above process of obtaining a preset neural network model can also be called a model training process.

[0061] Through the above-mentioned model training method, there is no need to obtain a large number of paired sample images in advance. Specifically, there is no need to obtain paired images of teeth to be repaired (such as images of uneven teeth) and images of teeth after repair (such as images of neat teeth), which can greatly save the cost of obtaining sample images. It can be understood that the number of existing paired sample images is small and the difficulty of obtaining them is usually high. If paired sample images are obtained by manual image editing, the required labor cost will be even higher. In contrast, the above-mentioned model acquisition method (also referred to as the model training method) provided in the embodiment of the present disclosure does not require the initial acquisition of paired sample images of teeth to be repaired and sample images of teeth after repair for model training. Instead, it first directly performs mouth area detection and lower teeth area detection on the first sample image with a higher degree of tooth beauty, and occludes the detected mouth area. By using the lower teeth area detection result to constrain the tooth structure in the generated image of the model to match the tooth structure of the first sample image, finally, based on the adversarial loss and pixel loss, the tooth beauty and realism of the generated image of the model can be effectively guaranteed.

[0062] For ease of understanding, based on FIG2 , we can further refer to a flowchart of an image processing method shown in FIG3 , which mainly includes the following steps:

[0063] Step S302: Acquire a first image to be processed.

[0064] Step S304 : Detect the mouth region of the target object in the first image, and generate a mouth region mask image based on the detected mouth region.

[0065] Step S306 , generating a second image according to the mouth region mask image and the first image; wherein the pixel values ​​of the mouth region in the second image are all designated pixel values.

[0066] Step S308 : detecting the lower teeth region of the target object in the first image to obtain a lower teeth detection result.

[0067] Step S310: Perform restoration processing based on the lower teeth detection result and the second image to obtain a target image.

[0068] By using the above-mentioned method of first blocking and then repairing the mouth area, the teeth of the target object in the image can be adjusted conveniently and effectively, which can better meet the needs of users. Moreover, during the repair process, due to the constraints of the lower teeth detection results, the tooth structure in the target image finally obtained matches the tooth structure in the first image, and there will be no situation where there are no lower teeth in the first image but there are lower teeth in the target image, or there are lower teeth in the first image but there are no lower teeth in the target image, etc., which effectively guarantees the reliability of the target image finally generated and comprehensively guarantees the user experience. For ease of understanding, the embodiment of the present disclosure further provides a schematic diagram of the effect of repairing the mouth area using the above-mentioned image processing method, as shown in Figure 4, which illustrates that uneven teeth can be adjusted to neat teeth, with a good tooth beautification effect, effectively improving the user's perception.

[0069] It should be noted that Figures 2 to 4 are all illustrated using the example of the target area being the mouth area and the designated sub-area within the target area being the lower teeth area. In actual applications, the target area and its designated sub-area can also be, for example, a tree trunk and its hole, a flower and its stamens, an eye and its pupil, etc., and the specific settings can be flexible and are not limited here. The above-mentioned method provided by the embodiments of the present disclosure can conveniently and effectively adjust the target area in the image, which can better meet user needs.

[0070] Corresponding to the aforementioned image processing method, FIG5 is a schematic structural diagram of an image processing device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. As shown in FIG5 , the image processing device includes:

[0071] An image acquisition module 502 is configured to acquire a first image to be processed;

[0072] The region blocking module 504 is used to detect the target region in the first image and block the detected mouth region to obtain a second image;

[0073] The image restoration module 506 is configured to perform restoration processing on the target image based on the second image to obtain the target image; wherein the target area in the target image is different from the target area in the first image.

[0074] The technical solution provided by the disclosed embodiments can detect a target area in a first image to be processed, occlude the detected mouth area to obtain a second image, and further perform restoration processing based on the second image to obtain a target image, wherein the target area in the target image differs from the target area in the first image. By first occluding the target area and then restoring the target area, the target area in the image can be adjusted conveniently and effectively, which can better meet the user's needs for adjusting the target area in the image.

[0075] In some embodiments, the image restoration module 506 is specifically used to: detect a specified sub-region in the target region in the first image to obtain a sub-region detection result; and perform restoration processing based on the sub-region detection result and the second image to obtain a target image.

[0076] In some embodiments, the designated subregion in the target image is matched with the designated subregion in the first image.

[0077] In some embodiments, the sub-region detection result is presented in the form of a specified sub-region mask image; the region repair module 506 is specifically used to: input the specified sub-region mask image and the second image into a preset neural network model, so that the neural network model repairs the target area in the second image based on the sub-region detection result.

[0078] In some embodiments, a model acquisition module is also included, which is used to obtain a neural network model according to the following steps: obtaining a first sample image; detecting the target area in the first sample image, and occluding the detected target area to obtain a second sample image; obtaining a designated sub-area mask image corresponding to the first sample image, and inputting the second sample image and the designated sub-area mask images corresponding to the first sample image into the initial neural network model to obtain a generated image output by the initial neural network model; adjusting the parameters of the initial neural network model based on the generated image, the first sample image and a preset loss function until a neural network model that meets preset conditions is obtained.

[0079] In some embodiments, the aesthetic level of the target area in the first sample image is greater than a preset threshold.

[0080] In some embodiments, the model acquisition module is further used to: obtain the adversarial loss value between the generated image and the first sample image based on a preset adversarial loss function; obtain the pixel loss value between the generated image and the first sample image based on a preset pixel loss function; and adjust the parameters of the initial network model according to the adversarial loss value and the pixel loss value.

[0081] In some embodiments, the area masking module 504 is specifically used to: generate a target area mask map based on the detected target area; generate a second image based on the target area mask map and the first image; wherein the pixel values ​​of the target area in the second image are all specified pixel values, and the pixel values ​​of the non-target area in the second image are consistent with the pixel values ​​of the non-target area in the first image.

[0082] The image processing device provided by the embodiments of the present disclosure can execute the image processing method provided by any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0083] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device embodiment can refer to the corresponding process in the method embodiment, and will not be repeated here.

[0084] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading the executable instructions from the memory and executing the instructions to implement the above-mentioned image processing method.

[0085] FIG6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG6 , the electronic device 600 includes one or more processors 601 and a memory 602 .

[0086] The processor 601 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 600 to perform desired functions.

[0087] The memory 602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 601 may execute the program instructions to implement the image processing method of the embodiment of the present disclosure described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0088] In one example, the electronic device 600 may further include an input device 603 and an output device 604 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0089] In addition, the input device 603 may also include, for example, a keyboard, a mouse, etc.

[0090] The output device 604 can output various information to the outside, including determined distance information, direction information, etc. The output device 604 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0091] Of course, for simplicity, FIG6 only shows some of the components related to the present disclosure in the electronic device 600, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 600 may further include any other appropriate components according to specific application scenarios.

[0092] In addition to the above-mentioned method and device, the embodiment of the present disclosure may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the image processing method provided by the embodiment of the present disclosure.

[0093] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0094] In addition, the embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor is enabled to execute the image processing method provided by the embodiment of the present disclosure.

[0095] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0096] The embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the image processing method in the embodiment of the present disclosure when executed by a processor.

[0097] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0098] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0099] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0100] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0102] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, comprising: Acquire a first image to be processed; Detecting a target area in the first image, and performing occlusion processing on the detected target area to obtain a second image; Performing restoration processing based on the second image to obtain a target image; wherein a target area in the target image is different from a target area in the first image.

2. The method according to claim 1, wherein the performing restoration processing based on the second image to obtain the target image comprises: Detecting a specified sub-region in the target region in the first image to obtain a sub-region detection result; A restoration process is performed based on the sub-region detection result and the second image to obtain a target image. 3 . The method of claim 2 , wherein a specified sub-region in the target image matches a specified sub-region in the first image.

4. The method according to claim 2, wherein the sub-region detection result is presented in the form of a specified sub-region mask image; and the repairing process based on the sub-region detection result and the second image comprises: The designated sub-region mask image and the second image are input into a preset neural network model, so that the neural network model performs repair processing on the target region in the second image based on the sub-region detection result.

5. The method according to claim 4, wherein the neural network model is obtained according to the following steps: Acquire a first sample image; Detecting the target area in the first sample image, and performing occlusion processing on the detected target area to obtain a second sample image; Obtaining a designated sub-region mask image corresponding to the first sample image, and inputting the designated sub-region mask images corresponding to the second sample image and the first sample image into an initial neural network model to obtain a generated image output by the initial neural network model; The parameters of the initial neural network model are adjusted based on the generated image, the first sample image and a preset loss function until a neural network model that meets preset conditions is obtained. 6 . The method according to claim 5 , wherein the aesthetic degree of the target area in the first sample image is greater than a preset threshold.

7. The method according to claim 5, wherein adjusting the parameters of the initial neural network model based on the generated image, the first sample image and a preset loss function comprises: Based on a preset adversarial loss function, obtaining an adversarial loss value between the generated image and the first sample image; Based on a preset pixel loss function, obtaining a pixel loss value between the generated image and the first sample image; According to the adversarial loss value and the pixel loss value, the parameters of the initial neural network model are adjusted.

8. The method according to claim 1, wherein the step of performing occlusion processing on the detected target area to obtain a second image comprises: Generating a target area mask map based on the detected target area; A second image is generated based on the target area mask image and the first image; wherein the pixel values ​​of the target area in the second image are all specified pixel values, and the pixel values ​​of the non-target area in the second image are consistent with the pixel values ​​of the non-target area in the first image.

9. An image processing device, comprising: An image acquisition module, used for acquiring a first image to be processed; A region shielding module, used for detecting a target region in the first image and performing shielding processing on the detected target region to obtain a second image; An image restoration module is used to perform restoration processing on a target image based on the second image to obtain a target image; wherein the target area in the target image is different from the target area in the first image.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image processing method described in any one of claims 1-8.

11. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image processing method according to any one of claims 1 to 8.

Images