A text-guided image editing method, apparatus, device, and medium

Abstract

The application provides a text-guided image editing method and device, equipment and medium, inputting an input picture, a source prompt word and a target prompt word into a model; the input picture is first added with noise to become a noise picture through an inversion process, and then respectively enters a reconstruction branch and an editing branch; a cross attention map of a reconstruction process of the reconstruction branch is injected into the editing branch to maintain the structure and content of the picture; a contrast attention focusing strategy is adopted in the inversion process; for different degrees of attention leakage, attention is accurately guided to the editing area adaptively; subsequently, a mask-guided mixed sampling technology is used in the editing branch to provide different conditional guidance for the editing area and the non-editing area, so as to ensure that the balance of changes and preservation in the image is integrated in the editing process; only the area needing editing is modified, unnecessary editing of the non-editing area is prevented, and the phenomenon of over-editing is prevented.

Description

Technical Field

[0001] This invention relates to the fields of image editing technology and deep learning, and particularly to a text-guided image editing method, apparatus, device, and medium. Background Technology

[0002] Large-scale text-based image diffusion models trained on massive datasets have made significant progress in generating high-fidelity and diverse images. Leveraging these models' text understanding capabilities, numerous methods for text-based image editing have emerged. Users can clarify their editing needs solely through text descriptions, without requiring any additional annotations.

[0003] P2P replaces the attention map of the target image with that of the source image to achieve image editing. Spatial features from the reconstruction stage are injected into specific layers of the editing process to preserve the structure of the input image. Masactrl implements mutual self-attention to achieve consistent image generation and non-rigid image editing. Pix2Pix-Zero identifies the editing direction by computing differences between text embeddings. InstructPix2Pix introduces an instruction-based image editing method. Hive incorporates human preferences and improves the performance of instruction-based image editing methods by designing a reward function.

[0004] Because image editing relies solely on text guidance, existing techniques often suffer from over-editing, meaning unnecessary modifications are made to non-editable areas of the input image. This over-editing occurs because previous image editing methods depended on coarse-grained text descriptions, making it difficult to precisely locate the target editing area. A deeper reason lies in attention leakage during the editing process, where cross-attention maps related to editing show high response values ​​in both target and non-target areas. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a text-guided image editing method, apparatus, device and medium that modifies only the area to be edited during the editing process, prevents unnecessary editing of non-editable areas, and prevents over-editing.

[0006] In a first aspect, the present invention provides a text-guided image editing method, which inputs an input image, source prompts, and target prompts into a text-generated image diffusion model; the source prompts are used to describe the content of the original image, and the target prompts are used to describe the content of the original image after editing.

[0007] The input image is first subjected to an inversion process to add noise, and then enters a reconstruction branch and an editing branch respectively. The reconstruction branch is used to reconstruct the input image, and the editing branch is used to edit the input image. The cross-attention map of the reconstruction process of the reconstruction branch is injected into the editing branch to maintain the structure and content of the image.

[0008] The inversion process employs a contrastive attention focusing strategy, which adaptively and precisely guides attention to the editing area for different degrees of attention leakage. Subsequently, the editing branch uses a mask-guided hybrid sampling technique to provide different condition guidance for the editing and non-editing areas, so as to ensure a balanced integration of changes and preservation in the image.

[0009] Secondly, the present invention provides a text-guided image editing device, comprising:

[0010] The input module is used to input the input image, source prompts, and target prompts into the text-to-image diffusion model; the source prompts are used to describe the content of the original image, and the target prompts are used to describe the content of the original image after editing.

[0011] The Inversion module is used to perform the inversion process on the input image, adding noise to create a noisy image.

[0012] The reconstruction branch module is used to reconstruct the input image;

[0013] The edit branch module is used to edit the input image;

[0014] The injection module is used to inject the cross-attention map of the reconstruction process of the reconstruction branch into the editing branch to maintain the structure and content of the image;

[0015] The contrast attention focusing strategy module is used to employ a contrast attention focusing strategy during the inversion process to adaptively and accurately guide attention to the editing area for different degrees of attention leakage.

[0016] The hybrid sampling module is used in the editing branch to employ mask-guided hybrid sampling techniques to provide different conditional guidance for the editing and non-editing areas, ensuring a balanced integration of changes and preservation in the image.

[0017] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0019] The present invention provides one or more technical solutions, which have at least the following technical effects or advantages: When the input image is annotated into a noisy image through the inversion process, the present invention employs a contrastive attention focusing strategy to adaptively and precisely guide attention to the editing area for different degrees of attention leakage. When the noisy image enters the reconstruction branch and the editing branch, the cross-attention map of the reconstruction branch is injected into the editing branch to maintain the image's structure and content. This enables precise text-guided image editing, modifying only the areas that need editing during the editing process and preventing unnecessary editing of non-editable areas. Experimental comparison results show that the present invention surpasses current technologies both quantitatively and qualitatively, especially in maintaining image structure and non-editable areas, where it is significantly superior to current technologies.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Figure 1 This is a schematic diagram of the system framework and process of the present invention;

[0023] Figure 2 This is a schematic diagram illustrating the qualitative comparison between embodiments of the present invention and other technologies;

[0024] Figure 3 This is a quantitative comparison data table between direct inversion and other technologies used in embodiments of the present invention;

[0025] Figure 4 This is a quantitative comparison data table of null-text inversion and negative-promptinversion with other techniques used in embodiments of the present invention;

[0026] Figure 5 This is a schematic diagram of the device in Embodiment 2 of the present invention; Detailed Implementation

[0027] This application provides a text-guided image editing method that modifies only the areas to be edited during the editing process, preventing unnecessary editing of non-editable areas and thus preventing over-editing.

[0028] The overall approach of the technical solution in this application is as follows: When the input image of the text-based image diffusion model is noisy after the inversion process, a contrastive attention focusing strategy is first adopted to adaptively and precisely guide attention to the editing area according to different degrees of attention leakage. When the noisy image enters the reconstruction branch and the editing branch, the cross-attention map of the reconstruction branch is injected into the editing branch to maintain the structure and content of the image. This enables precise text-guided image editing, modifying only the areas that need to be edited during the editing process, preventing unnecessary editing of non-editable areas.

[0029] Example 1

[0030] like Figure 1 As shown, this embodiment provides a text-guided image editing method, which inputs an input image, source prompts, and target prompts into a text-generated image diffusion model; the source prompts are used to describe the content of the original image, and the target prompts are used to describe the content of the original image after editing;

[0031] The input image is first subjected to an inversion process to add noise, and then enters a reconstruction branch and an editing branch respectively. The reconstruction branch is used to reconstruct the input image, and the editing branch is used to edit the input image. The cross-attention map of the reconstruction process of the reconstruction branch is injected into the editing branch to maintain the structure and content of the image.

[0032] The inversion process employs a contrastive attention focusing strategy, which adaptively and precisely guides attention to the editing area for different degrees of attention leakage. Subsequently, the editing branch uses a mask-guided hybrid sampling technique to provide different condition guidance for the editing and non-editing areas, so as to ensure a balanced integration of changes and preservation in the image.

[0033] Figure 1 In the image, the source prompt is "sourcepromptP: "an orange van with surfboards ontop"; the target prompt is "targetpromptP: "an orange van with flowers ontop". The noisy image is Z. T .

[0034] The aforementioned comparative attention-focusing strategy, which adaptively and precisely guides attention to the editing area for different degrees of attention leakage, specifically involves:

[0035] In the field of local image editing, an edit pair is extracted from the source cue and the target cue, denoted as...<key1,key2> The source prompt and the target prompt are in the format "P=..." <key1>···” and “P * =··· <key2>···";For example, Figure 1 The edit in the text consists of <surfboard, flowers>.

[0036] Since the content described in the source hint comes directly from the input image, "key1" in the source hint provides a more precise location of the edit region compared to "key2" mentioned in the target hint. Therefore, we will focus our efforts on the cross-attention map associated with "key1" to generate a new cross-attention map. The cross-attention map High response values ​​are only observed in the editing area;

[0037] In the edit branch, the original cross-attention map corresponding to "key2" is replaced with... To address the issue of attention leakage.

[0038] During the inversion process, the required cross-attention map is obtained by averaging attention maps of all heads and time steps in the inversion process at a resolution of 16*16. and self-attention map Enhanced cross-attention graph Contrast between the editing area and the area where attention leaks:

[0039] To avoid By focusing attention on non-editing areas, we enhance... The contrast between the editing region and the region where attention leakage occurs. First, compare the existing cross-attention map. Mini-max normalization is applied to generate a scaled graph s, where each element represents a cross-attention map. The magnitude of the corresponding element:

[0040]

[0041] Formula (1) is used to apply cross-attention maps Mapping to interval [s l ,s r ];s l Cross-attention map The left endpoint of the corresponding matrix, s r Cross-attention map The right endpoint of the corresponding matrix is ​​determined by the cross-attention map. Multiplying with the scaled map s amplifies the contrast between the response values ​​of the cross-attention map; this contrast enhancement effectively filters out irrelevant attention to non-editable areas while maintaining high attention to the areas to be edited.

[0042] Perform N contrast enhancement operations, and perform a normalization operation after each operation to restore the original value:

[0043]

[0044] Where j refers to the current number of contrast enhancement operations, j = 1 to N. N is determined based on the degree of attention leakage; the higher the degree of attention leakage, the larger N becomes. To determine the required number of contrast enhancement operations, we consider using the cosine similarity between text embeddings:

[0045]

[0046] Where 'e' is the reference text embedding, and 'e' is the reference text embedding. i Represents other text embeddings, where L is the total number of text tokens, and α adjusts the sensitivity of N to similarity.

[0047] Cross-attention map After performing N contrast enhancement operations, it is denoted as It effectively filters out unnecessary attention from non-editing areas and highlights the response values ​​for editing areas;

[0048] Using self-attention maps To restore the characteristics The shape of the high-response area in the text is adjusted to match the shape of the editing area; for and self-attention map Perform matrix multiplication to obtain

[0049]

[0050] After performing the operation of formula (4), The high-response area covers the editing area, but also slightly increases the attention to the non-editing area;

[0051] Subsequently, in order to eliminate Pay attention to non-editing areas, and further enhance its attention to editing areas. Perform attention enhancement operations:

[0052]

[0053] Where k represents each element in the iterative cross attention graph; τ1 represents the threshold, which can be set empirically.

[0054] right After performing the attention enhancement operation, attention values ​​below τ1 are set to zero, and attention values ​​above τ1 are further enhanced.

[0055] This invention also proposes a novel sampling technique, namely mask-guided hybrid sampling, which limits the editing action to the target editing area.

[0056] The mask-guided hybrid sampling technique, which provides different condition guidance for edited and non-edited areas, specifically includes:

[0057] Replace the non-editable regions of the latent code for the edited branch with the corresponding non-editable regions of the latent code for the reconstructed branch:

[0058]

[0059] in, This represents the potential code obtained from denoising at each step of the edit branch, where m represents the mask of the edit region. This represents the potential code obtained from each step of the denoising process in the reconstruction branch;

[0060] Given The high-response region accurately contains the edit region, using a threshold τ2 from Extract the mask m:

[0061]

[0062] τ2 is set based on experience;

[0063] The mask-guided hybrid sampling of this invention provides different conditional guidance tailored to both editable and non-editable regions; ensuring that editable regions follow the target prompts while keeping non-editable regions in their original state.

[0064]

[0065] Among them, c s and c t These represent the text embeddings of the source and target prompt words, respectively; g s and g t represents the intensity of the conditional guidance applied to the non-editing region and the editing region, respectively, and m represents the mask of the editing region.

[0066] like Figure 2 As shown, four examples are implemented using the method of this invention. The first column is the input image, the second column uses the existing Pix2Pix-Zero method, the third column uses the existing MasaCtrl method, the fourth column uses the existing PnP method, the fifth column uses the existing P2P method, and the sixth column uses the method of this invention. It can be seen that, compared to various existing methods, this invention achieves the best results in ensuring that the editing area follows the target prompts while maintaining the original state of the non-editing area.

[0067] like Figure 3 and Figure 4 As shown, for various methods of the present invention relative to existing technologies, quantitative comparative data on the use of direct inversion in the embodiments of the present invention compared with other technologies can be found in [link to relevant documentation]. Figure 3 For quantitative comparison data between null-text inversion and negative-prompt inversion and other techniques used in embodiments of the present invention, please refer to [link to relevant documentation]. Figure 4 .

[0068] Example 2

[0069] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0070] like Figure 5 As shown, this embodiment provides a text-guided image editing device, including:

[0071] The input module is used to input the input image, source prompts, and target prompts into the text-to-image diffusion model; the source prompts are used to describe the content of the original image, and the target prompts are used to describe the content of the original image after editing.

[0072] The Inversion module is used to perform the inversion process on the input image, adding noise to create a noisy image.

[0073] The reconstruction branch module is used to reconstruct the input image;

[0074] The edit branch module is used to edit the input image;

[0075] The injection module is used to inject the cross-attention map of the reconstruction process of the reconstruction branch into the editing branch to maintain the structure and content of the image;

[0076] The contrast attention focusing strategy module is used to employ a contrast attention focusing strategy during the inversion process to adaptively and accurately guide attention to the editing area for different degrees of attention leakage.

[0077] The hybrid sampling module is used in the editing branch to employ mask-guided hybrid sampling techniques to provide different conditional guidance for the editing and non-editing areas, ensuring a balanced integration of changes and preservation in the image.

[0078] In the domain of local image editing, the contrastive attention focusing strategy module extracts an edit pair from the source cue and the target cue, denoted as...<key1,key2> The source prompt and the target prompt are in the format "P=..." <key1>"..." and "P*=..." <key2>···”;

[0079] Focus your efforts on the cross-attention graph related to "key1" to generate a new cross-attention graph. The cross-attention map High response values ​​are only observed in the editing area;

[0080] The hybrid sampling module replaces the original cross-attention map corresponding to "key2" in the edit branch with... To address the issue of attention leakage.

[0081] The attention-focusing strategy module specifically performs the following process:

[0082] During the inversion process, the required cross-attention map is obtained by averaging attention maps of all heads and time steps in the inversion process at a resolution of 16*16. and self-attention map Enhanced cross-attention graph Contrast between the editing area and the area where attention leaks:

[0083] First, we examine the existing cross-attention graph. Mini-max normalization is applied to generate a scaled graph s, where each element represents a cross-attention map. The magnitude of the corresponding element:

[0084]

[0085] Formula (1) is used to apply cross-attention maps Mapping to interval [s l ,s r ];s l Cross-attention map The left endpoint of the corresponding matrix, s r Cross-attention map The right endpoint of the corresponding matrix is ​​determined by the cross-attention map. Multiplying by the scaled image s amplifies the contrast between the response values ​​of the cross-attention map;

[0086] Perform N contrast enhancement operations, and perform a normalization operation after each operation to restore the original value:

[0087]

[0088] Where j refers to the current number of contrast enhancement operations, j = 1 to N. N is determined by the cosine similarity between text embeddings:

[0089]

[0090] Where 'e' is the reference text embedding, and 'e' is the reference text embedding. i Represents other text embeddings, where L is the total number of text tokens, and α adjusts the sensitivity of N to similarity.

[0091] Cross-attention map After performing N contrast enhancement operations, it is denoted as

[0092] Using self-attention maps To restore the characteristics The shape of the high-response area in the text is adjusted to match the shape of the editing area; for and self-attention map Perform matrix multiplication to obtain

[0093]

[0094] After performing the operation of formula (4), The high-response area covers the editing area, but also slightly increases the attention to the non-editing area;

[0095] Subsequently, Perform attention enhancement operations:

[0096]

[0097] Where k represents each element in the iterative cross attention graph; τ1 represents the threshold, which is set empirically.

[0098] right After performing the attention enhancement operation, attention values ​​below τ1 are set to zero, and attention values ​​above τ1 are further enhanced.

[0099] The hybrid sampling module specifically performs the following process:

[0100] Replace the non-editable regions of the latent code for the edited branch with the corresponding non-editable regions of the latent code for the reconstructed branch:

[0101]

[0102] in, This represents the potential code obtained from denoising at each step of the edit branch, where m represents the mask of the edit region. This represents the potential code obtained from each step of the denoising process in the reconstruction branch;

[0103] Given The high-response region accurately contains the edit region, using a threshold τ2 from Extract the mask m:

[0104]

[0105] τ2 is set based on experience;

[0106] Mask-guided hybrid sampling provides tailored guidance for different conditions in the editable and non-editable areas; ensuring that the editable area follows the target prompts while keeping the non-editable area in its original state.

[0107]

[0108] Among them, c s and c t These represent the text embeddings of the source and target prompt words, respectively; g s and g t represents the intensity of the conditional guidance applied to the non-editing region and the editing region, respectively, and m represents the mask of the editing region.

[0109] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0110] Example 3

[0111] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3. This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement any of the implementation methods in Embodiment 1.

[0112] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.

[0113] Example 4

[0114] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4. This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement any of the implementation methods in Embodiment 1.

[0115] The technical solutions provided in this application embodiment have at least the following technical effects or advantages:

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A text-guided image editing method, characterized in that: inputting an input picture, a source prompt word and a target prompt word into a text-to-image diffusion model; the source prompt word is used to describe the content of the original picture, and the target prompt word is used to describe the content of the edited picture; the input picture is first added with noise as a noise picture through an inversion process, and then enters a reconstruction branch and an editing branch respectively; the reconstruction branch is used to reconstruct the input image, and the editing branch is used to edit the input image; a cross attention map of a reconstruction process of the reconstruction branch is injected into the editing branch to maintain the structure and content of the picture; the inversion process adopts a contrast attention focusing strategy, and adaptively guides the attention to the editing area accurately for different degrees of attention leakage; subsequently, a mask-guided mixed sampling technology is used in the editing branch to provide different conditional guidance for the editing area and the non-editing area to ensure the balanced integration of changes and preservation in the image; The required cross attention map is obtained by averaging the attention maps of resolution 16*16 of all heads and time steps in the inversion process and self-attention maps ; enhancement contrast between the edited region and the region where attention leakage occurs: First, the existing cross-attention graph Applying min-max normalization to generate a scaled graph s, each element in the scaled graph s represents the degree of magnitude of the corresponding element in （1） Equation (1) is used to map the cross-attention map to the interval ; where the left end point of the cross-attention map corresponds to the matrix, where the right end point of the cross-attention map corresponds to the matrix, by multiplying the cross-attention map with the scaling map s, which amplifies the contrast between the cross-attention map response values. Perform Subcontrast-enhancement operations, with normalization after each operation to restore original value size: （2） wherein, a number of times of the contrast enhancement operation currently performed, =1~N, , determined by cosine similarity between text embeddings: （3） wherein, is a reference text embedding, while denotes other text embeddings, is the number of total text tokens, adjusts the sensitivity to similarity; Cross-attention diagram Performing Post-contrast enhancement operation noted as ; Using self-attention maps To restore the characteristics The shape of the high-response area in the text is adjusted to match the shape of the editing area; for and self-attention map Perform matrix multiplication to obtain : （4） After the operation of formula (4) is performed, The high response area of the edit region is covered, but also slightly increases the attention to the non-edit region; Subsequently, the attention enhancement operation is performed on the attention enhanced operation: （5） wherein, represents iterating through each element in the cross-attention graph; represents a threshold, set empirically; Attention values below are set to zero and attention values above are further enhanced after attention enhancement operations are performed. ​ the mask-guided mixed sampling technology provides different conditional guidance for the editing area and the non-editing area, specifically: the non-editing area of the editing branch latent code is replaced with the corresponding non-editing area of the reconstruction branch latent code: （6） wherein, a mask representing an editing region, In view of the high response region accurately contains the edit region, the threshold is extracted from the mask : （7） wherein Set empirically; the mask-guided mixed sampling provides different conditional guidance for the editing area and the non-editing area; it ensures that the editing area follows the target prompt, while keeping the non-editing area in the original state: （8） wherein, and respectively represent text embeddings of source and target cue words; and respectively represent the strength of the conditional guidance applied to the non-editing region and the editing region, represents a mask of the editing region.

2. The text-guided image editing method of claim 1, wherein: the contrast attention focusing strategy adaptively guides the attention to the editing area accurately for different degrees of attention leakage, specifically: In the field of local image editing, an editing pair is extracted from the source cue and the target cue, denoted as <key1, key2>, and the formats of the source cue and the target cue are =· <key1>"and" =· ·· <key2> · · ·”；< / key2> Focus on the cross-attention map related to "key1" to generate a new cross-attention map The cross-attention map Has high response values only in the edit region; In the editing branch, the original cross-attention graph corresponding to "key2" is replaced by to solve the problem of attention leakage.

3. An image editing apparatus based on text guidance, characterized by: including: an input module, configured to input an input picture, a source prompt word and a target prompt word into a text-to-image diffusion model; the source prompt word is used to describe the content of the original picture, and the target prompt word is used to describe the content of the edited picture; an inversion module, configured to perform an inversion process on the input picture to add noise as a noise picture; a reconstruction branch module, configured to reconstruct the input image; an editing branch module, configured to edit the input image; an injection module, configured to inject a cross attention map of a reconstruction process of the reconstruction branch into the editing branch to maintain the structure and content of the picture; a contrast attention focusing strategy module, configured to adopt a contrast attention focusing strategy in the inversion process, and adaptively guide the attention to the editing area accurately for different degrees of attention leakage; a mixed sampling module, configured to use a mask-guided mixed sampling technology in the editing branch to provide different conditional guidance for the editing area and the non-editing area to ensure the balanced integration of changes and preservation in the image; the contrast attention focusing strategy module specifically performs the following process: The required cross attention map is obtained by averaging the attention maps of resolution 16*16 of all heads and time steps in the inversion process and self-attention maps ; enhancement contrast between the edited region and the region where attention leakage occurs: First, the existing Applying min-max normalization to generate a scaled map s, each element in the scaled map s represents the degree of magnitude of the corresponding element in the middle （1） Equation (1) is used to map the cross-attention map to the interval ; denotes the left end point of the cross-attention map corresponding matrix, denotes the right end point of the cross-attention map corresponding matrix, by multiplying the cross-attention map with the scaling map s, the contrast between the cross-attention map response values is amplified. Perform Subcontrast-enhancement operations, with normalization after each operation to restore original value size: （2） wherein, a number of times of the contrast enhancement operation currently performed, = 1 ~ N, , determined by cosine similarity between text embeddings: （3） wherein, is a reference text embedding, while denotes other text embeddings, is the number of total text tokens, adjusts the sensitivity to similarity; Cross-attention diagram Performing Post-contrast enhancement operation noted as ; Restoring the shape of a high response region in a self-attention map to be consistent with the shape of the edit region; and :​​​ （4） After the operation of formula (4) is performed, The high response area of the edit region is covered, but also slightly increases the attention to the non-edit region; Subsequently, Perform attention enhancement operations: （5） wherein, represents each element in the iterative cross-attention map; represents a threshold, which is set empirically; right After performing attention enhancement operations, the result will be lower than The attention value is set to zero, and further enhanced to a value higher than zero. Attention value; the mixed sampling module specifically performs the following process: the non-editing area of the editing branch latent code is replaced with the corresponding non-editing area of the reconstruction branch latent code: （6） wherein, a mask representing an editing region, In view of the high response region accurately contains the edit region, the threshold is extracted from the mask : （7） wherein Set by experience; the mask-guided mixed sampling provides different conditional guidance for the editing area and the non-editing area; it ensures that the editing area follows the target prompt, while keeping the non-editing area in the original state: （8） wherein, and respectively represent text embeddings of source and target cue words; and respectively represent the strength of the conditional guidance applied to the non-editing region and the editing region, represents a mask of the editing region.

4. The text-guided image editing apparatus of claim 3, wherein: The contrast attention focusing strategy module extracts an editing pair from the source prompt and the target prompt in the local image editing field, denoted as <key1, key2>, and the formats of the source prompt and the target prompt are respectively =··· <key1>"and" =· · · <key2> · · ·”；< / key2> Focus on the cross-attention map related to "key1" to generate a new cross-attention map The cross-attention map Has high response values only in the edit region; The mixed sampling module is in the editing branch, the original "key2" corresponding cross attention graph is replaced by To solve the problem of attention leakage.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, the processor executes the program to implement the method of any one of claims 1-2.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the method of any one of claims 1 to 4.

Images