An image inpainting method, device, computer equipment and storage medium

Abstract

The present disclosure provides an image repairing method and device, computer equipment and a storage medium. The present disclosure discloses obtaining a heat map carrying at least one response area, wherein the heat map is generated by using original labels of an original image, each response area in the heat map corresponds to an original label, and the original label is obtained by content understanding of the original image. Target repairing guide information corresponding to the original label is obtained, and a target generation model corresponding to the target repairing guide information is obtained. The heat map and the target repairing guide information are input into the target generation model, and the target generation model is used to repair the response area in the heat map by using target features corresponding to the target repairing guide information, so as to obtain a target image. The present disclosure realizes automatic content repairing according to content understanding of an image. The whole process does not need a user to input repairing content and adjust model parameters, and the repairing efficiency of the image is improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, specifically to an image restoration method, apparatus, computer device, and storage medium. Background Technology

[0002] Current image inpainting solutions, while possessing classification models and local image restoration techniques across various fields, still require manual intervention during the restoration process. For example, in common image generation techniques, the local image drawing function still requires users to manually select the areas in the image that need restoration, manually input the restoration content, and select the diffusion generation model that produces the best results. This process necessitates continuous adjustments to the restoration content and a large number of model parameters to achieve satisfactory generation results.

[0003] It is evident that existing image restoration solutions are cumbersome, time-consuming, and cannot achieve automated restoration. Summary of the Invention

[0004] In view of this, the present disclosure provides an image restoration method, apparatus, computer device, and storage medium to solve the problems that existing image restoration schemes are cumbersome to operate, time-consuming, and unable to achieve automated restoration.

[0005] In a first aspect, embodiments of this disclosure provide an image restoration method, characterized in that the method includes:

[0006] A heatmap carrying at least one response region is obtained, wherein the heatmap is generated using the original labels of the original image, and each response region in the heatmap corresponds to an original label, the original label being obtained by content understanding of the original image;

[0007] Obtain the target repair guidance information corresponding to the original label, and obtain the target generation model corresponding to the target repair guidance information;

[0008] The heatmap and the target repair guidance information are input into the target generation model. The target generation model uses the target features corresponding to the target repair guidance information to repair the response area in the heatmap to obtain the target image.

[0009] Secondly, embodiments of this disclosure provide an image restoration apparatus, the apparatus comprising:

[0010] An acquisition module is used to acquire a heatmap carrying at least one response region, wherein the heatmap is generated using the original labels of the original image, each response region in the heatmap corresponds to an original label, and the original label is obtained by content understanding of the original image;

[0011] The processing module is used to obtain the target repair guidance information corresponding to the original label, and to obtain the target generation model corresponding to the target repair guidance information;

[0012] The repair module is used to input the heatmap and the target repair guidance information into the target generation model, and use the target features corresponding to the target repair guidance information to repair the response area in the heatmap to obtain the target image.

[0013] Thirdly, embodiments of this disclosure provide a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method described in the first aspect or any corresponding embodiment.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first aspect or any corresponding embodiment.

[0015] The method provided in this disclosure utilizes original tags obtained through content understanding of the original image to acquire corresponding repair guidance information, and uses the target generation model corresponding to the repair guidance information to repair the response areas in the heatmap, thus realizing automatic content repair based on image content understanding. The entire process does not require users to input repair content or adjust model parameters, improving image repair efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of an image restoration method provided according to some embodiments of the present disclosure;

[0018] Figure 2 These are schematic diagrams of original images provided according to some embodiments of this disclosure;

[0019] Figure 3 This is a schematic diagram of the heat map corresponding to the original image provided according to some embodiments of this disclosure;

[0020] Figure 4 This is a schematic diagram of the structure of a target generation model provided according to some embodiments of this disclosure;

[0021] Figure 5 This is a schematic diagram of a heatmap after feature removal according to some embodiments of this disclosure;

[0022] Figure 6 This is a schematic diagram of an initial repair image provided according to some embodiments of this disclosure;

[0023] Figure 7 This is a schematic diagram of an image restoration process provided according to some embodiments of the present disclosure;

[0024] Figure 8 This is a structural block diagram of an image restoration apparatus provided according to some embodiments of the present disclosure;

[0025] Figure 9 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] According to embodiments of this disclosure, an image restoration method, apparatus, computer device, and storage medium are provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0028] This embodiment provides an image restoration method. Figure 1 This is a flowchart of an image restoration method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the process includes the following steps:

[0029] Step S11: Obtain a heatmap carrying at least one response region, wherein the heatmap is generated using the original labels of the original image, and each response region in the heatmap corresponds to an original label, which is obtained by content understanding of the original image.

[0030] The method provided in this disclosure is applied to a processing device with image processing capabilities, such as a computer, smartphone, smartwatch, etc. The processing device can acquire the original image to be processed, generate a heatmap carrying response regions based on the original image, match corresponding repair guidance information according to the original labels corresponding to the response regions, and finally perform image repair according to the repair guidance information to obtain the final target image. This achieves automatic image repair; the entire image repair process requires no manual adjustment by the user, nor does it require the user to configure corresponding guidance information, thus improving image repair efficiency.

[0031] In this embodiment of the disclosure, obtaining a heatmap carrying at least one response region includes the following steps A1-A4:

[0032] Step A1: Perform content understanding on the original image to be processed to obtain entity information and semantic information in the original image.

[0033] In this embodiment of the disclosure, content understanding of the original image to be processed can be performed using a content understanding model to obtain entity information and semantic information in the original image. The content understanding model can be a classification model, a semantic recognition model, etc. Entity information can be entity objects carried in the original image, such as animals, people, plants, buildings, etc. Semantic information can be the image style in the original image, such as cartoon style, realistic style, etc.

[0034] As an example, the original image is as follows: Figure 2 As shown, the original image is input into the classification model. The model extracts the image features from the original image and classifies them based on these features to obtain the entity information in the original image. The entity information is as follows: Figure 2 Examples include cats, apples, and trees. The original image can also be input into a semantic recognition model, which extracts the image style features of the original image. Based on these features, the image style of the original image is determined to be cartoon style, which is then used as the semantic information of the original image.

[0035] Step A2: Construct the original labels based on entity information and semantic information.

[0036] In this embodiment of the disclosure, the original label constructed based on entity information and semantic information can be: label: "Cat, Apple, Tree, Cartoon...".

[0037] Step A3: Extract target image features from the original image that match the original label.

[0038] In this embodiment of the disclosure, obtaining target image features that match the original label from the original image includes: extracting image features from the original image; calculating the confidence level of the image features relative to the original label; and determining the image features with a confidence level greater than or equal to a preset confidence level as target image features.

[0039] In this embodiment, the original image is input into a feature detection model. The model first extracts image features from the original image and calculates the similarity between the image features and the corresponding label features of the original label. This similarity is used as the confidence level of the original image relative to the original label. Finally, image features with a confidence level greater than or equal to a preset confidence level are determined as target image features. It should be noted that a higher confidence level of an image feature relative to the original label indicates a better match between the image feature and the original label; conversely, a lower confidence level indicates a mismatch. Finally, image features with a confidence level greater than or equal to the preset confidence level are determined as target image features.

[0040] Step A4: Draw the response region based on the location of the target image features in the original image to obtain a heatmap.

[0041] In this embodiment of the disclosure, the feature points corresponding to the target image features in the original image are first determined, and a feature region is constructed based on the feature points. Then, the confidence level corresponding to the target image features in the feature region is calculated, and the mean confidence level is used as the confidence level of the feature region. Finally, the feature region in the original image is rendered, and the confidence level corresponding to the feature region is marked on the original feature to obtain a heatmap.

[0042] As an example, image features are extracted from the original image, and the confidence scores of these features relative to the original labels are calculated. This determines the feature regions for the entity "cat" and the uniquely located regions for the entity "apple" in the original image. The confidence scores for the feature regions corresponding to "cat" and "apple" are calculated to be 0.98 and 0.99 respectively. Finally, the feature regions corresponding to "cat" and "apple" are rendered, and the confidence scores for each feature region are marked on the image, resulting in a final drawing as shown below. Figure 3 The heatmap shown.

[0043] Step S12: Obtain the target repair guidance information corresponding to the original label, and obtain the target generation model corresponding to the target repair guidance information.

[0044] In this embodiment of the disclosure, obtaining the target repair guidance information corresponding to the original tag includes the following steps B1-B2:

[0045] Step B1: Obtain the mapping relationship between preset tags and corpus information.

[0046] In this embodiment, the mapping relationship between corpus information in preset tags is obtained from the corpus. The mapping relationship between the preset tags and corpus information can be: apple—watermelon, dog—duck, strawberry—rose, cartoon—real, etc. The mapping relationship between the preset tags and corpus information can be pre-configured. Subsequent users can customize the mapping relationship between the preset tags and corpus information according to their repair needs.

[0047] Step B2: Determine the target corpus information corresponding to each original label from the mapping relationship, and generate target repair guidance information based on the target corpus information.

[0048] In this embodiment of the disclosure, the target repair guidance information (prompt) generated based on the target corpus information is "watermelon, real,...". Here, "watermelon" and "real" are the target corpus information corresponding to the original tags.

[0049] The method provided in this disclosure establishes a mapping relationship between preset tags and corpus information, eliminating the need for users to manually input repair content during the image restoration process. It can directly match the corresponding corpus information based on the original tags obtained from the content understanding of the image. Compared with manual input of repair content, this method saves a lot of operation steps and provides a foundation for the automation of content understanding and content restoration.

[0050] In this embodiment of the disclosure, obtaining the target generation model corresponding to the target repair guidance information includes the following steps C1-C2:

[0051] Step C1: Obtain the target conditional control network corresponding to each target corpus information in the target repair guidance information from the model library. The model library includes multiple conditional control networks corresponding to multiple corpus information. The conditional control networks are trained based on the conditional constraint information corresponding to the corpus information.

[0052] In this embodiment, the model library includes multiple pre-trained conditional control networks. Each conditional control network corresponds to conditional constraint information for a given corpus information. The conditional constraint information can be that entities within the image maintain consistent contours or colors before and after restoration, etc. Based on this, after determining the target restoration guidance information, the corresponding target conditional control network is matched from the model library using the target prediction information it carries.

[0053] In this embodiment of the disclosure, the training method of the conditional control network includes: obtaining training sample images using conditional constraint information corresponding to corpus information; and training an initial conditional control network using the training sample images to obtain the conditional control network.

[0054] As an example, taking an apple as the entity, when the conditional constraint is contour consistency, the training sample images include: a first image carrying the apple and a second image carrying the apple's contour. The second image is obtained by contour detection of the apple in the first image. The first image is input into an initial conditional control network, which outputs a predicted contour of the apple in the first image. The predicted contour is then compared with the actual contour in the second image to calculate the training loss. The initial conditional control network is optimized based on the training loss until the predicted contour output by the initial conditional control network matches the actual contour. Alternatively, when the conditional constraint is color consistency, the training sample images include: the original image and its corresponding black and white image. A conditional control network for color constraints is trained using the original image and its corresponding black and white image to ensure that the entity colors are consistent before and after image restoration.

[0055] Step C2: Construct a target generation model based on the preset generation network and the target condition control network.

[0056] In this embodiment of the disclosure, a target generation model is obtained by superimposing a preset generation network and a target condition control network. The structure of the target generation model is as follows: Figure 4 As shown.

[0057] It should be noted that traditional image inpainting schemes also lack specific constraint information during the diffusion generation process, resulting in deviations between the inpainted image and the original image, which affects the quality of image inpainting.

[0058] Based on this, the method provided in this disclosure utilizes the corpus information corresponding to the original labels to match a conditional control network, and then superimposes the conditional control network with the generator network to obtain the final target generation model. Subsequently, the target generation model is directly used to repair the original image, eliminating the need for manual adjustment of the model parameters and improving the model repair efficiency. Simultaneously, by combining the generator network with the conditional control network, the quality of image repair can be effectively guaranteed, solving the problem of deviation between the generated image and the desired image.

[0059] Step S13: Input the heat map and target repair guidance information into the target generation model. The target generation model uses the target features corresponding to the target repair guidance information to repair the response area in the heat map to obtain the target image.

[0060] In this embodiment of the disclosure, the target image is obtained by repairing the response region in the heatmap using the target features corresponding to the target repair guidance information through the target generation model, including the following steps D1-D4:

[0061] Step D1: Input the heatmap into the preset generation network and the condition control network respectively.

[0062] Step D2: Using a preset generation network, the response area of ​​the heatmap is repaired by utilizing the target features corresponding to the target corpus information to obtain an initial repaired image.

[0063] In this embodiment of the disclosure, the preset generation network has a local rendering function. Therefore, the preset generation network can remove features from the response area in the heatmap, and then add the target features corresponding to the target corpus information to the response area to obtain the initial repaired image.

[0064] Step D3: Extract feature information from the response region of the heatmap that matches the conditional constraint information corresponding to the target corpus information using a conditional control network;

[0065] In this embodiment of the disclosure, a conditional control network extracts corresponding feature information from the response region of the heatmap according to the conditional constraint information corresponding to the target corpus. The feature information may be features such as contours and colors.

[0066] Step D4: Fuse the initial repaired image and feature information to obtain the target image.

[0067] As an example, with Figure 3 The heatmap shown illustrates this, with the response region representing the area containing the word "apple." The corresponding corpus information for this response region is "watermelon." Based on this, the heatmap is input into both the preset generation network and the conditional control network in the target generation model. The preset generation network removes the "apple" feature from the response region (the removed image is shown below). Figure 5 As shown), then add the target features corresponding to "watermelon" to the response region to obtain the initial repaired image (as shown). Figure 6 (As shown). An apple contour is extracted from the response region of the heatmap using a conditional control network according to the conditional constraints corresponding to the target corpus. Finally, the apple contour is fused with the target features corresponding to the watermelon in the initial restoration image to ensure that the watermelon contour in the final target image is consistent with the apple contour. This effectively guarantees the quality of the image restoration.

[0068] The method provided in this disclosure utilizes original tags obtained through content understanding of the original image to acquire corresponding repair guidance information, and uses the target generation model corresponding to the repair guidance information to repair the response areas in the heatmap, thus realizing automatic content repair based on image content understanding. The entire process does not require users to input repair content or adjust model parameters, improving image repair efficiency.

[0069] Figure 7 This is a schematic diagram of an image restoration process according to an embodiment of the present disclosure, such as... Figure 7 As shown, the repair process includes the following steps:

[0070] (1) Content comprehension stage:

[0071] Step 1.1: Input the original image into the content understanding model. The content understanding model will classify and semantically recognize the original image to obtain entity information and semantic information in the original image.

[0072] Step 1.2: Construct the original labels corresponding to the original image using entity information and semantic information. The original labels are mainly: Cat, Apple, Cartoon.

[0073] (2) Heatmap generation process:

[0074] Step 2.1: Obtain target image features that match the original labels from the original image, and draw response regions based on the positions of the target image features in the original image to obtain a heatmap.

[0075] (3) Image restoration process:

[0076] Step 3.1: Obtain the mapping relationship between preset tags and corpus information, determine the target corpus information corresponding to each original tag from the mapping relationship, and generate target repair guidance information based on the target corpus information. The target repair guidance information is: Dog, Watermelon, Real.

[0077] Step 3.2: Obtain the target condition control network corresponding to each target corpus information in the target repair guidance information from the model library, and construct the target generation model based on the preset generation network and the target condition control network.

[0078] Step 3.3: Input the heatmap into the preset generation network and the conditional control network respectively. The preset generation network uses the target features corresponding to the target corpus information to generate the heatmap. The conditional control network extracts feature information from the response area of ​​the heatmap that matches the conditional constraint information corresponding to the target corpus information. Finally, the initial repaired image and the feature information are fused to obtain the target image.

[0079] This embodiment also provides an image restoration apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0080] This embodiment provides an image restoration device, such as... Figure 8 As shown, it includes:

[0081] The acquisition module 81 is used to acquire a heatmap carrying at least one response region, wherein the heatmap is generated using the original labels of the original image, and each response region in the heatmap corresponds to an original label, which is obtained by content understanding of the original image.

[0082] Processing module 82 is used to obtain the target repair guidance information corresponding to the original label and to obtain the target generation model corresponding to the target repair guidance information;

[0083] Repair module 83 is used to input the heat map and target repair guidance information into the target generation model. The target generation model uses the target features corresponding to the target repair guidance information to repair the response area in the heat map to obtain the target image.

[0084] In this embodiment of the disclosure, the acquisition module 81 is used to perform content understanding on the original image to be processed, and obtain entity information and semantic information in the original image; construct original labels based on entity information and semantic information; obtain target image features that match the original labels from the original image; and draw response regions according to the positions of the target image features in the original image to obtain a heat map.

[0085] In this embodiment of the disclosure, the acquisition module 81 is used to extract image features from the original image; calculate the confidence level of the image features relative to the original label; and determine the image features with a confidence level greater than or equal to a preset confidence level as target image features.

[0086] In this embodiment of the disclosure, the processing module 82 is used to obtain the mapping relationship between preset tags and corpus information; determine the target corpus information corresponding to each original tag from the mapping relationship, and generate target repair guidance information from the target corpus information.

[0087] In this embodiment of the disclosure, the processing module 82 is used to obtain the target condition control network corresponding to each target corpus information in the target repair guidance information from the model library. The model library includes multiple condition control networks corresponding to multiple corpus information. The condition control networks are trained based on the condition constraint information corresponding to the corpus information. A target generation model is constructed based on the preset generation network and the target condition control network.

[0088] In this embodiment of the present disclosure, the apparatus further includes: a training module, configured to acquire training sample images using conditional constraint information corresponding to the corpus information; and to train an initial conditional control network using the training sample images to obtain a conditional control network.

[0089] In this embodiment, the repair module is used to input the heatmap into a preset generation network and a conditional control network respectively; repair the response area of ​​the heatmap using the target features corresponding to the target corpus information through the preset generation network to obtain an initial repaired image; extract feature information from the response area of ​​the heatmap that matches the conditional constraint information corresponding to the target corpus information through the conditional control network; and fuse the initial repaired image and the feature information to obtain the target image.

[0090] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this disclosure, such as... Figure 9 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0091] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0092] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0093] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0094] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0095] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0096] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0097] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An image inpainting method characterized by, The method includes: A heatmap carrying at least one response region is obtained, wherein the heatmap is generated using the original labels of the original image, and each response region in the heatmap corresponds to an original label, the original label being constructed from entity information and semantic information in the original image obtained by content understanding of the original image to be processed; Obtain the target repair guidance information corresponding to the original label, and obtain the target generation model corresponding to the target repair guidance information. The target generation model is constructed based on a preset generation network and the target condition control network corresponding to the target repair guidance information. The target condition control network is located in the model library, which includes multiple condition control networks. Each condition control network corresponds to a condition constraint information corresponding to a corpus information. The condition constraint information includes that the entities in the image maintain the same outline and color before and after repair. The heatmap and the target repair guidance information are input into the target generation model. The target generation model uses the target features corresponding to the target repair guidance information to repair the response area in the heatmap to obtain the target image.

2. The method of claim 1, wherein, The step of obtaining a heatmap carrying at least one response region includes: Content understanding is performed on the original image to be processed to obtain entity information and semantic information in the original image; The original label is constructed based on the entity information and the semantic information; Obtain target image features that match the original label from the original image; The response region is drawn based on the location of the target image features in the original image to obtain the heatmap.

3. The method of claim 2, wherein, The step of obtaining target image features from the original image that match the original label includes: Extract image features from the original image; Calculate the confidence level of the image features relative to the original label, and determine the image features whose confidence level is greater than or equal to a preset confidence level as the target image features.

4. The method of claim 1, wherein, The step of obtaining the target repair guidance information corresponding to the original tag includes: Obtain the mapping relationship between preset tags and corpus information; The target corpus information corresponding to each original tag is determined from the mapping relationship, and the target repair guidance information is generated based on the target corpus information.

5. The method of claim 4, wherein, The step of obtaining the target generation model corresponding to the target repair guidance information includes: Obtain the target condition control network corresponding to each target corpus information in the target repair guidance information from the model library, wherein the model library includes multiple condition control networks corresponding to multiple corpus information, and the condition control network is trained based on the condition constraint information corresponding to the corpus information; The target generation model is constructed based on the preset generation network and the target condition control network.

6. The method of claim 5, wherein, The training method for the conditional control network includes: The training sample images are obtained using the conditional constraint information corresponding to the corpus information. The initial conditional control network is trained using the training sample images to obtain the conditional control network.

7. The method according to claim 5, characterized in that, The step of repairing the response region in the heatmap using the target features corresponding to the target repair guidance information through the target generation model to obtain the target image includes: The heatmaps are input into the preset generation network and the condition control network, respectively. The preset generation network uses the target features corresponding to the target corpus information to repair the response region of the heatmap, thereby obtaining an initial repaired image; The conditional control network extracts feature information from the response region of the heatmap that matches the conditional constraint information corresponding to the target corpus information. The target image is obtained by fusing the initial repaired image and the feature information.

8. An image restoration device, characterized in that, The device includes: An acquisition module is used to acquire a heatmap carrying at least one response region, wherein the heatmap is generated using the original labels of the original image, each response region in the heatmap corresponds to an original label, and the original label is constructed from entity information and semantic information in the original image obtained by content understanding of the original image to be processed; The processing module is used to obtain the target repair guidance information corresponding to the original label and obtain the target generation model corresponding to the target repair guidance information. The target generation model is constructed based on a preset generation network and the target condition control network corresponding to the target repair guidance information. The target condition control network is located in the model library, which includes multiple condition control networks. Each condition control network corresponds to the condition constraint information corresponding to a corpus information. The condition constraint information includes that the entities in the image maintain the same outline and color before and after repair. The repair module is used to input the heatmap and the target repair guidance information into the target generation model, and use the target features corresponding to the target repair guidance information to repair the response area in the heatmap to obtain the target image.

9. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.

Images