Emotion image editing method and system based on multi-agent cooperation framework

Abstract

The invention provides an emotion image editing method and system based on a multi-agent cooperation framework, and the method comprises the steps: a pre-creation stage: generating a preliminary editing scheme based on a pre-obtained source image and target emotion through a planning agent, and adjusting the preliminary editing scheme according to an evaluation result through an evaluation agent, then executing a preliminary editing operation through an editing agent according to the adjusted preliminary editing scheme to obtain a preliminary editing image; in the optimization stage, whether the preliminary editing image effectively conveys the target emotion or not is evaluated through an evaluation agent, and an optimization editing instruction is generated according to the evaluation result that the target emotion is not effectively conveyed; and the editing agent iteratively updates the preliminary editing image according to the optimization editing instruction until the target emotion is effectively conveyed. According to the method, efficient and accurate emotion-oriented image editing can be realized, more flexible image editing and emotion expression means are provided, and the method is suitable for various image processing and creative generation tasks.

Description

Technical Field

[0001] The present invention relates to the field of image editing under the field of artificial intelligence technology, and more specifically, to an emotional image editing method and system based on a multi-agent collaboration framework. Background Art

[0002] Image editing has always been an important means for people to process images and express aesthetics. Compared with the current semantic image editing methods, emotional image editing can understand and realize abstract emotional expression, aiming to adjust the visual elements in the image from the emotional dimension to evoke special emotional feelings. With the continuous development of deep learning technology, multimodal technology and visual language models, emotional image editing has become a more valuable research field because it can provide more accurate emotional expression, more in line with human cognition and more artistic editing results.

[0003] However, existing emotional image editing methods mainly focus on low-level and simple editing methods, such as color and style, which cannot achieve full and accurate emotional expression. Images in the real world usually use a variety of expression techniques to express specific emotions; in addition, some work, based on semantic image editing, converts emotions into simple codes or other features to guide image editing. However, given the complexity of human emotional expression, this often cannot fully and accurately express the target emotion and is prone to image structure distortion. In addition, by converting emotional image editing into a multi-step semantic image editing task, methods such as EmoEdit can enhance the target emotional expression by changing the visual elements in the image. However, EmoEdit cannot reasonably and accurately handle the complex interactions and connections between various semantic and emotional elements and various editing methods.

[0004] Therefore, how to ensure the rationality and effectiveness of image editing results in expressing the target emotion has become one of the current research directions of emotional image editing tasks. Summary of the invention

[0005] In view of the problem that the current emotional image editing methods are unable to achieve reasonable and effective target emotional expression, the purpose of the present invention is to provide an emotional image editing method and system based on a multi-intelligent collaborative body framework, which will utilize multiple intelligent agents to simulate the cognitive behavior of human painters and be responsible for the three functions of decision-making, editing, and evaluation to collaboratively complete the emotional image editing task, so as to enhance the interpretability and reliability of the operation results.

[0006] The emotional image editing method based on the multi-agent collaborative framework provided by the present invention includes a pre-creation stage and an optimization stage; wherein: In the pre-creation stage, a preliminary editing scheme is generated by a planning agent based on the acquired source image and target emotion; according to the preliminary editing scheme, the preliminary editing scheme is evaluated by an evaluation agent, and the preliminary editing operation in the preliminary editing scheme is performed by an editing agent to obtain a preliminary edited image; In the optimization stage, the preliminary edited image is evaluated by an evaluation agent. If the preliminary edited image does not meet the target emotion, an optimization editing instruction is generated, and the preliminary edited image is adjusted according to the optimization editing instruction by an editing agent.

[0007] Among them, an optional solution is that the pre-creation stage includes: S111: analyzing the emotion and semantics of the source image by a planning agent to obtain the emotion information and semantic information of the source image; S112: Based on the target emotion, retrieve emotion editing elements related to the target emotion from an emotion editing element database through a preset emotion factor knowledge retriever; S113: generating a preliminary editing scheme of the emotional image according to the emotional editing elements, wherein the preliminary editing includes specific editing operations and adjustment sequence; S114: Evaluating the preliminary editing plan through an evaluation agent, and optimizing the preliminary editing plan according to the evaluation result to form an optimized editing plan; S115: The editing agent performs the image editing task according to the optimized editing scheme to generate a preliminary edited image.

[0008] Among them, an optional solution is that the optimization stage includes: S121: Evaluating the preliminary editing result through an evaluation agent to determine whether the preliminary editing result effectively conveys the target emotion; wherein, if the preliminary editing result fails to achieve the expected effect, the evaluation agent provides an optimization editing instruction for the preliminary editing result; S122: The editing agent iteratively updates the preliminary editing result according to the optimized editing instruction to obtain an optimized edited image.

[0009] Among them, an optional scheme is that the planning intelligent agent determines each editing element and specific steps of the editing operation in the preliminary editing plan through a three-layer decision tree structure; wherein the three-layer decision tree takes the target emotion as the root node, identifies the appropriate editing elements as branch nodes, and assigns the corresponding editing methods as leaf nodes.

[0010] Among them, an optional solution is that the emotional factor knowledge retriever uses a method based on retrieval enhancement generation to retrieve image elements related to the target emotion from a preset emotional data set to generate the preliminary editing solution.

[0011] Among them, an optional solution is that, in the pre-creation stage, the editing agent gradually implements the contents of all editing instructions in the optimized editing scheme on the source image in an iterative manner according to the optimized editing scheme; wherein an image editing algorithm tool library is integrated in the editing agent, and in the image editing algorithm tool library, different editing methods correspond to different image editing algorithms.

[0012] Among them, an optional solution is to loop the evaluation, adjustment processing and re-editing operations of the optimization stage for the editing results of the pre-creation stage until the evaluation agent obtains an evaluation result that satisfies the expression of the target emotion.

[0013] The present invention also provides a multi-agent collaboration framework for implementing the above-mentioned emotional image editing method, comprising: a planning agent for receiving a target emotion and a source image and generating a preliminary editing scheme according to the source image and the target emotion; An editing agent, configured to edit the source image according to the preliminary editing scheme so that the edited image meets the requirements of the target emotion in terms of vision and emotion; An evaluation agent is used to evaluate the preliminary editing plan and the editing results edited according to the preliminary editing plan, and to put forward optimization suggestions based on the evaluation results to form an optimized editing plan; and to evaluate the edited image, determine whether it effectively conveys the target emotion, and put forward optimization suggestions for the editing results that do not meet the standards.

[0014] The present invention also provides an emotional image editing system based on a multi-agent collaborative framework, which uses the emotional image editing method based on a multi-agent collaborative framework as described above to edit emotional images, including a pre-creation unit and an optimization unit; wherein, The pre-creation unit is used to generate a preliminary editing scheme based on the acquired source image and target emotion through a planning agent; according to the preliminary editing scheme, the preliminary editing scheme is evaluated by an evaluation agent, and the preliminary editing operation of the preliminary editing scheme is performed by an editing agent to obtain a preliminary edited image; The optimization unit is used to evaluate the preliminary edited image through an evaluation agent, obtain an optimization edit instruction according to the evaluation result, and adjust the preliminary edited image according to the optimization edit instruction through an editing agent.

[0015] The present invention further provides an electronic device, comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the steps in the emotional image editing method based on the multi-agent collaboration framework as described above.

[0016] It can be seen from the above technical scheme that the emotional image editing method and system based on the multi-agent collaborative framework provided by the present invention realizes accurate image editing driven by emotion by combining the emotional editing element retrieval and multi-agent collaboration through the user input of the source image and the target emotional description. First, in the pre-creation stage, the source image is subjected to emotional and semantic analysis, and the relevant emotional editing elements are retrieved through the preset emotional factor knowledge retriever to generate a preliminary emotional image editing plan. Then, the evaluation agent evaluates the effectiveness of the preliminary editing plan, puts forward optimization suggestions, and optimizes the editing plan on this basis, and finally generates an optimized editing plan. Then, the editing agent performs preliminary editing operations according to the optimized plan to generate preliminary editing results. In the optimization stage, the evaluation agent evaluates whether the preliminary editing results effectively convey the target emotion, and generates optimized editing instructions based on the evaluation feedback. The editing agent iteratively generates updated images according to these instructions until the image successfully conveys the target emotion. The technical scheme of the present invention effectively realizes emotion-driven image editing, improves the accuracy and flexibility of image emotional expression, and is suitable for various creative image generation and emotion-oriented image editing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings: Figure 1 A flowchart of an emotional image editing method based on a multi-agent collaboration framework according to an embodiment of the present invention; Figure 2 is an overview diagram of three agents according to an embodiment of the present invention; Figure 3 A schematic diagram of a workflow of an emotional image editing method based on a multi-agent collaboration framework according to an embodiment of the present invention; Figure 4 A schematic diagram of a three-layer decision tree structure of a planning agent according to an embodiment of the present invention; Figure 5 A schematic diagram of an action space in an editing agent according to an embodiment of the present invention; Figure 6A schematic diagram of a system framework for emotional image editing based on a multi-agent collaboration framework according to an embodiment of the present invention; Figure 7 is a schematic diagram of an electronic device according to an embodiment of the present invention; Figure 8 and Fig. 9 A schematic diagram showing the comparison of the effects of applying the present invention and other existing image editing solutions; Fig.10 and Fig.11 They are two example diagrams of the effects of image editing using the emotional image editing method and system based on a multi-agent collaborative framework of the present invention. DETAILED DESCRIPTION

[0018] In the following description, for the purpose of illustration, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details. In other examples, for ease of describing one or more embodiments, known structures and devices are shown in the form of block diagrams.

[0019] In response to the problems of inaccurate emotion expression and lack of flexibility in the editing process in existing emotion image editing methods, the present invention proposes an emotion image editing method and system based on a multi-agent collaborative framework, which combines emotion needs with source images and utilizes the multi-agent collaborative framework to perform image editing and optimization so as to more accurately convey the target emotion. Among them, the present invention can effectively solve the problem of inconsistency between emotion understanding and editing instruction execution in traditional emotion image editing methods through an emotion factor knowledge retriever and a refined editing scheme generation process, and significantly improve the quality and accuracy of image editing effects.

[0020] In order to better illustrate the technical solution of the present invention, some technical terms involved in the present invention are briefly explained below.

[0021] Image editing is the process of making highly sophisticated and detailed modifications and adjustments to an image. In image editing tasks, a source image and related editing instructions and reference images are usually given to generate a target image that meets the editing requirements and retains the characteristics of the original image. The core challenge of this task is how to effectively balance the editing effect and the fidelity to the original content, so as to ensure that the final image can meet the user's specific editing requirements while retaining the core visual attributes of the original image.

[0022] An intelligent agent is an intelligent entity built on a large language model that has the ability to perceive the environment, understand autonomously, make decisions, and execute actions. An intelligent agent can simulate independent thinking processes, flexibly call on various tools, and gradually achieve preset goals. It aims to complete complex tasks through the close combination of perception, thinking, and action.

[0023] CoT (Chain of Thought) is a technology that breaks down logically complex problems into a series of multi-step thinking processes with logical relationships, and explains the entire thinking process by showing a small number of examples to a large language model, thereby activating the thinking ability of the large model to answer more complex and difficult questions.

[0024] Prompt Engineering improves the performance of large language models by designing and improving their prompts. The goal is to create an effective and controllable large language model system that can perform specific tasks more accurately and reliably.

[0025] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] It should be noted that the following description of the exemplary embodiments is merely illustrative and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the technologies and devices should be considered as part of the specification.

[0027] In order to illustrate the emotional image editing method and system based on the multi-agent collaboration framework provided by the present invention, Figures 1 to 5 The process and principle of the emotional image editing method based on the multi-agent collaboration framework according to the embodiment of the present invention are respectively exemplarily indicated; Figure 6 The logical structure of the emotional image editing system based on the multi-agent collaboration framework of the embodiment of the present invention is exemplarily indicated.

[0028] like Figures 1 to 5 As shown in the figure, the emotional image editing method based on the multi-agent collaboration framework provided by the present invention mainly includes the following steps: S110: Pre-creation stage, through multi-agent multi-step processing, a preliminary editing plan is generated, and preliminary editing operations are performed, and finally a preliminary edited image is obtained.

[0029] Specifically, in the pre-creation stage, a preliminary editing plan is generated by a planning agent based on the acquired source image and target emotion; according to the preliminary editing plan, the preliminary editing plan is evaluated by an evaluation agent, and the preliminary editing operation of the preliminary editing plan is executed by an editing agent to obtain a preliminary edited image.

[0030] S120: Optimization stage, obtaining optimized editing instructions through evaluation and feedback of the preliminary edited image, and adjusting the preliminary editing result according to the optimized editing instructions to better convey the target emotion.

[0031] Specifically, in the optimization stage, the preliminary edited image is evaluated by an evaluation agent. If the preliminary edited image does not meet the target emotion, an optimization editing instruction is generated, and the preliminary edited image is adjusted according to the optimization editing instruction by the editing agent; if it meets the target emotion, the preliminary edited image is directly output.

[0032] Specifically, as an example, the planning agent analyzes the emotion and semantics of the source image through a multimodal language model to obtain the emotion information and semantic information of the source image. The multimodal language model based on the present invention can adopt Openai's GPT series multimodal model. For the convenience of expression, in the following embodiments, GPT4o is used as an implementation example of the multimodal language model.

[0033] The above steps of the emotional image editing method based on the multi-agent collaboration framework will be described in more detail below in combination with specific embodiments.

[0034] More specifically, as an example, for step S110, the specific method of obtaining the preliminary edited image in the pre-creation stage is as follows: For the source image entered by the user , target emotion First, the planning agent analyzes the source image through a multimodal language model to obtain its semantic information (semantic text description ) and emotional information (raw emotion ); then, based on the semantic information and emotional information of the source image, the planning agent uses a preset emotional factor knowledge retriever to retrieve appropriate emotional editing elements, and then determines each editing element and specific steps of the editing operation in the preliminary editing scheme through a three-layer decision tree structure. The three-layer decision tree is based on the target emotional As the root node, select the element with high text similarity As a branch node, edit mode As a leaf node, and finally determined as an edit instruction .

[0035] After multiple rounds of retrieval, the preliminary editing plan is integrated. The evaluation agent will evaluate the target emotion expression degree of the preliminary editing plan and make corrections to the editing instructions that cannot fully express the target emotion. Modify the editing elements in the Transmit to the editing agent for specific editing operations to obtain the edited image in the pre-creation stage , that is, preliminarily edit the image.

[0036] The pre-creation stage proposed in the present invention is equivalent to saying that the emotion control of the input source image is a large framework, and the single emotion image editing task performs the first image editing within this large framework. Specifically, the pre-creation stage includes the following steps: S111: The planning agent analyzes the emotion and semantics of the source image to obtain the emotion information and semantic information of the source image; wherein the emotion analysis mainly focuses on the specific original emotion in the source image, and the emotion feature of the source image is obtained through the emotion analysis, and the emotion feature is the specific emotion element contained in the source image; the semantic analysis focuses on the semantic features such as objects and scenes in the image; S112: Based on the acquired target emotion, the planning agent retrieves the emotion editing elements through a preset emotion factor knowledge retriever; specifically, based on the target emotion, the emotion factor knowledge retriever is used to retrieve the emotion editing elements related to the target emotion from the emotion editing element database; Among them, the emotional factor knowledge retriever can use a method based on retrieval enhancement generation to retrieve image elements related to the target emotion from a preset emotional data set to generate the preliminary editing plan.

[0037] S113: Generate a detailed emotional image editing plan; specifically, based on the emotional editing elements retrieved from the emotional factor knowledge retriever, generate a detailed preliminary editing plan for the emotional image, and the preliminary editing plan is used to list specific editing operations, adjustment sequence, etc. to ensure that the image can be transformed toward the target emotion.

[0038] In a specific embodiment of the present invention, based on the semantic features of the image, the original emotion and the target emotion, appropriate visual elements and editing methods are selected to generate a preliminary editing plan, including searching in an emotional factor knowledge retriever, and using a large-scale language model to independently conceive editing elements that best match the image semantics and the target emotion, and selecting an appropriate editing method based on the editing elements and the image semantics. The algorithm for generating the preliminary editing plan is as follows:

[0039] in, represents the i-th editing instruction, Represents the element to be edited and edit mode The editing elements include: independent objects, scenes, object attributes, colors, filters, and facial expressions; the editing methods include: adding, replacing, deleting, expression editing, filter editing, background editing, and attribute editing.

[0040] Among them, an optional solution is that the emotional factor knowledge retriever includes constructing an emotional factor knowledge base and constructing an emotional factor retrieval engine. Among them, the emotional factor knowledge base is constructed based on the emotional data set EmoSet, each data contains descriptive text that describes a specific editing element in detail and is marked with its corresponding emotion and element type; the emotional factor retrieval engine retrieves the descriptive text of the emotional element with the highest semantic similarity to the original image semantic text description.

[0041] S114: Optimizing the preliminary editing plan; specifically, evaluating the preliminary editing plan through an evaluation agent to determine whether the preliminary editing plan can effectively convey the target emotion, and optimizing the preliminary editing plan according to the evaluation result, and finally forming an optimized editing plan; Among them, evaluating the preliminary editing plan includes using a multimodal language model to understand the emotion expressed by the image to determine whether it is consistent with the target emotion. Optimizing suggestions for the preliminary editing plan based on the evaluation results include obtaining new editing instructions by updating editing elements, and adjusting editing instructions that are not executed correctly and editing instructions that fail to successfully express the target emotion to new editing instructions and instructing the editing agent to re-edit.

[0042] S115: Perform preliminary editing operations; specifically, the editing agent performs image editing tasks according to the above-mentioned optimized editing scheme to generate preliminary edited images. These image editing tasks may include color adjustment, light and shadow changes, expression modification, object addition, object removal, object replacement, object attribute editing, background editing, etc. Through these editing operations, a preliminary edited image is generated.

[0043] Among them, an optional scheme is that the above-mentioned image editing task includes adopting different image editing algorithms for the editing methods of each editing instruction to perform specific editing operations according to the given editing instructions. Among them, the specific editing operations include pre-processing node operations, editing node operations, and post-verification node operations; the pre-processing node operations include using algorithms such as image detection and segmentation to generate target masks to assist the image editing algorithm in performing editing operations. Among them, the editing node operations include using image editing algorithms to perform specific editing operations to obtain edited images; the post-verification nodes include using visual language models to verify and compare the edited images with the editing instructions, and re-editing the edited images that do not comply with the editing instructions.

[0044] It should be noted that, in the pre-creation stage, the editing agent can, according to the optimized editing scheme, implement the contents of all editing instructions in the optimized editing scheme on the source image step by step in an iterative manner; wherein, an image editing algorithm tool library is integrated in the editing agent, and in the image editing algorithm tool library, different editing methods correspond to different image editing algorithms, and the editing methods include: object addition, object replacement, object removal, expression editing, background editing, filter editing, and object attribute editing.

[0045] For step S120, the final specific method obtained through the optimization stage is as follows: Regarding the preliminary editing result described in step S110 , evaluate the intelligent experience of expressing the target emotion The effectiveness and rationality of the editing results are evaluated. For the editing results that can reasonably and effectively express the target emotion, this round of emotional image editing task is completed.

[0046] For editing results that cannot reasonably and effectively express the target emotion, the evaluation agent will evaluate the editing instructions that were not executed correctly. Optimize and adjust the data, and then re-transmit it to the editing agent for editing. The optimization and adjustment mainly includes re-executing the editing instructions. to perform an editing operation, or readjust the text description of the editing instruction.

[0047] The optimization stage proposed in the present invention is equivalent to performing subsequent image editing for a single emotional image editing task within the entire emotional image editing framework. Specifically, the optimization stage may further include the following steps: S121: Evaluate the preliminary editing results and give optimized editing instructions; specifically, the evaluation agent evaluates the preliminary editing results to determine whether the preliminary editing results effectively convey the target emotion. If the preliminary editing results fail to achieve the expected effect, the evaluation agent will provide detailed optimized editing instructions and point out the parts that need further adjustment.

[0048] S122: Iteratively generate an updated image according to the optimized editing instruction; specifically, the editing agent adjusts the preliminary editing result according to the optimized editing instruction to generate an updated image. Here, steps S121 to S122 are looped, that is, the preliminary editing result is iteratively updated until the updated image can effectively convey the target emotion.

[0049] In a specific embodiment of the present invention, the above-mentioned optimization process cycle can be executed i times to obtain the final editing result. .

[0050] From the overall framework of emotional image editing, the entire process of implementing emotional image editing in the present invention includes two process branches: a pre-creation phase and an optimization phase. and target emotion To generate preliminary editing results , the optimization phase uses the preliminary editing results Gradually adjust the target emotion expression, generating the expression that can reasonably and effectively express the target emotion The final result .

[0051] In other words, the pre-creation phase and the optimization phase are two iterative editing phases, and the optimization phase requires the preliminary editing results of the pre-creation phase. The cyclic optimization phase is to adjust the editing plan and edit again so that the final editing result can more reasonably express the target emotion. For the two process branches, each agent is set with its own workflow and tasks, and the setting operation can adopt the techniques of Prompt Engineering and CoT.

[0052] It can be seen from the above embodiments that the multi-agent collaboration framework for implementing the above-mentioned emotional image editing method mainly includes three parts: planning agent, editing agent and evaluation agent.

[0053] Among them, the planning agent is used to receive the target emotion and the source image and generate a preliminary editing plan according to the source image and the target emotion; the editing agent is used to edit the source image according to the preliminary editing plan generated by the planning agent, so that the edited image meets the requirements of the target emotion in terms of vision and emotion; the evaluation agent is used to evaluate the preliminary editing plan and the editing results edited according to the preliminary editing plan, and put forward optimization suggestions based on the evaluation results to form an optimized editing plan. Among them, if the editing result edited according to the preliminary editing plan does not meet the target emotion, an optimized editing instruction is generated; if it meets the target emotion, the preliminary edited image can be directly output; and the evaluation agent is also used to evaluate the edited image, determine whether it effectively conveys the target emotion, and put forward optimization suggestions for the editing results that do not meet the standards.

[0054] In addition, the present invention also provides an emotional image editing system based on a multi-agent collaborative framework, which uses the emotional image editing method based on the multi-agent collaborative framework as described above to perform emotional image editing. The emotional image editing system based on the multi-agent collaborative framework mainly includes a pre-creation unit and an optimization unit; wherein the pre-creation unit is used to generate a preliminary editing plan based on the acquired source image and target emotion through a planning agent, and based on the preliminary editing plan, evaluate the preliminary editing plan through an evaluation agent, and execute the preliminary editing operation in the preliminary editing plan through an editing agent to obtain a preliminary edited image; the optimization unit is used to evaluate the preliminary edited image through an evaluation agent, obtain an optimized editing instruction based on the evaluation result, and adjust the preliminary edited image according to the optimized editing instruction through the editing agent.

[0055] From another perspective, the emotional image editing system 300 based on the multi-agent collaboration framework provided by the present invention is Figure 6 As shown, it mainly includes three parts: editing generation unit 310, editing execution unit 320, and editing evaluation unit 330.

[0056] The editing generation unit 310 is used to generate content of an editing solution that meets user needs based on user input data; wherein the user input data includes a source image and a target emotion; An editing execution unit 320, used to perform actual editing on the content of the editing scheme to obtain an edited image; The editing evaluation unit 330 is used to evaluate and verify the edited image in terms of emotion and semantic dimensions.

[0057] The editing and generating unit 310 may further include: The sentiment analysis unit 311 is used to analyze the semantic text description and sentiment information of the source image input by the user; An emotional element retrieval unit 312, configured to retrieve appropriate reference emotional elements based on the target emotion and the semantic text description of the source image; The editing scheme decision unit 313 is used to conceive appropriate editing elements and contents based on the semantic text description and target emotion of the source image, and formulate an editing scheme matching the source image in combination with the reference emotion elements.

[0058] The editing execution unit 320 may further include: An editing tool library unit 321, used to integrate different types of image editing algorithms to complete actual editing operations, wherein the image editing algorithm includes a semantic editing algorithm; An editing preprocessing unit 322 is used to perform corresponding preprocessing operations on the source image and the intermediate editing result to assist in completing the actual editing, wherein the preprocessing operations include: target detection and segmentation, pixel fine-grained optimization, etc.; The editing decision unit 323 is used to receive the editing plan and autonomously call a suitable image editing algorithm in the editing tool library unit to complete all editing operations to obtain the final edited image.

[0059] The editing and evaluation unit 330 may further include: The verification and evaluation unit 331 is used to complete the emotion and semantic verification and evaluation of the edited image or editing scheme to obtain the target emotion matching degree and semantic suitability.

[0060] The editing optimization unit 332 is used to optimize the editing scheme for the edited image that does not achieve effective target emotional expression to enhance the target emotional expression of the image.

[0061] The above-mentioned emotion image editing system based on the multi-agent collaboration framework is an implementation system corresponding to the above-mentioned emotion image editing control method based on the multi-agent collaboration framework. Its specific execution steps can refer to the specific implementation example of the above-mentioned emotion image editing method based on the multi-agent collaboration framework, and will not be described in detail here.

[0062] It can be seen from the above embodiments that the emotional image editing method and system based on the multi-agent collaboration framework proposed by the present invention realizes accurate image editing driven by emotion through user input of source image and target emotional description, and combines emotional editing element retrieval with multi-agent collaboration. First, in the pre-creation stage, a preliminary emotional image editing plan is generated by performing emotional and semantic analysis on the source image, and retrieving relevant emotional editing elements through the emotional factor knowledge retriever; then, the evaluation agent evaluates the effectiveness of the preliminary editing plan, puts forward optimization suggestions, and optimizes the editing plan on this basis, and finally generates an optimized editing plan; then, the editing agent performs preliminary editing operations according to the optimized plan to generate preliminary editing results; in the optimization stage, the evaluation agent evaluates whether the preliminary editing results effectively convey the target emotion, and generates optimized editing instructions based on the evaluation feedback, and the editing agent iteratively generates updated images according to these instructions until the image successfully conveys the target emotion. Emotion-driven image editing is effectively realized, and the accuracy and flexibility of image emotional expression are improved.

[0063] Figures 8 to 11 The image editing effects of the emotional image editing method based on the multi-agent collaborative framework of the present invention are shown respectively. Figure 8 , Fig. 9This is a schematic diagram comparing the effects of applying the present invention and other existing image editing solutions. Fig.10 and Fig.11 The present invention provides two examples of the effects of image editing using the emotional image editing method and system based on a multi-agent collaborative framework.

[0064] like Figure 8 , Fig. 9 As shown, the text in the lower part of each line is the target emotion of the image conversion required by the user, including eight types.

[0065] The first column of images are original images, the last column (the ninth column) image "ours" represents the result of image editing using the emotional image editing method and system based on the multi-agent collaborative framework of the present invention, and the second to the eighth columns are the result of other comparison schemes. Other comparison schemes are as follows: IP2P: Paper InstructPix2Pix: Learning to Follow Image EditingInstructions; MagicBrush: Paper MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing; PnP: Paper Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation; Insdiff: Paper Instructdiffusion: A Generalist Modeling Interface for Vision Tasks; CLVA: Language-Driven Artistic Style Transfer; AIF: Affective Image Filter: Reflecting Emotions from Text to Images; EmoEdit*: Modeled after the paper Emoedit: Evoking Emotions through ImageManipulation.

[0066] exist Fig.10In the figure, every two pictures form a group, the left side is the original picture, and the right side is the editing effect picture of the image edited by applying the emotional image editing method and system based on the multi-agent collaborative framework of the present invention. The text below the right image is the target emotion.

[0067] exist Fig.11 In the figure, every nine images form a group. The leftmost one is the original image, and the eight images on the right are the editing effect images of the image editing method and system based on the multi-agent collaborative framework of the present invention. The text under each image on the right is the target emotion, showing that the emotional image editing method and system based on the multi-agent collaborative framework can convert the image emotion into any target emotion.

[0068] like Figure 7 As shown, the present invention also provides an electronic device, the electronic device comprising: at least one processor; and, a memory communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the steps in the aforementioned emotional image editing method based on a multi-agent collaborative framework.

[0069] Those skilled in the art will appreciate that the structure shown in the figure does not limit the electronic device 1 and may include fewer or more components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0070] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0071] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0072] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0073] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0074] The emotional image editing program 12 based on the multi-agent collaboration framework stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, the following steps can be implemented: S110: Pre-creation stage, through multi-agent multi-step processing, a preliminary editing plan is generated, and preliminary editing operations are performed, and finally a preliminary edited image is obtained; S120: Optimization stage, obtaining optimized editing instructions through evaluation and feedback of the preliminary edited image, and adjusting the preliminary editing result according to the optimized editing instructions to better convey the target emotion.

[0075] The multi-step process of the S110 pre-creation phase includes: S111: The planning agent analyzes the emotion and semantics of the source image to obtain the emotion information and semantic information of the source image; wherein the emotion analysis mainly focuses on the emotion features in the source image, and the semantic analysis focuses on the semantic features such as objects and scenes in the image; S112: Based on the acquired target emotion, the planning agent retrieves the emotion editing elements through a preset emotion factor knowledge retriever; specifically, based on the target emotion, the emotion factor knowledge retriever is used to retrieve the emotion editing elements related to the target emotion from the emotion editing element database; Among them, the emotional factor knowledge retriever can use a method based on retrieval enhancement generation to retrieve image elements related to the target emotion from a preset emotional data set to generate the preliminary editing plan.

[0076] S113: Generate a detailed emotional image editing plan; specifically, based on the emotional editing elements retrieved from the emotional factor knowledge retriever, generate a detailed preliminary editing plan for the emotional image, and the preliminary editing plan is used to list specific editing operations, adjustment sequence, etc. to ensure that the image can be transformed toward the target emotion.

[0077] S114: Optimizing the preliminary editing plan; specifically, evaluating the preliminary editing plan through an evaluation agent to determine whether the preliminary editing plan can effectively convey the target emotion, and optimizing the preliminary editing plan according to the evaluation result, and finally forming an optimized editing plan; Among them, evaluating the preliminary editing plan includes using the visual language model to understand the emotion expressed by the image to determine whether it is consistent with the target emotion. Optimizing the preliminary editing plan based on the evaluation results includes obtaining new editing instructions by updating the editing elements, and adjusting the editing instructions that were not executed correctly and the editing instructions that did not successfully express the target emotion to new editing instructions and instructing the editing agent to re-edit.

[0078] S115: Perform preliminary editing operations; specifically, the editing agent performs image editing tasks according to the above-mentioned optimized editing scheme to generate preliminary edited images. These image editing tasks may include color adjustment, light and shadow changes, expression modification, object addition, object removal, object replacement, object attribute editing, background editing, etc. Through these editing operations, a preliminary edited image is generated.

[0079] S120 is a multi-step process in the optimization stage, including: S121: evaluating the preliminary editing results and giving optimization editing instructions; specifically, the preliminary editing results are evaluated by an evaluation agent to determine whether the preliminary editing results effectively convey the target emotion. If the preliminary editing results fail to achieve the expected effect, the evaluation agent will provide detailed optimization editing instructions and point out the parts that need further adjustment.

[0080] S122: Iteratively generate an updated image according to the optimized editing instructions; specifically, the editing agent adjusts the preliminary editing results according to the above-mentioned optimized editing instructions to generate an updated image. Here, steps S121 to S122 are looped, that is, the preliminary editing results are iteratively updated until the updated image can effectively convey the target emotion. Specifically, the specific implementation method of the processor 10 for the above-mentioned instructions can refer to the description of the corresponding embodiment in the emotional image editing method of the multi-agent collaboration framework, which will not be repeated here.

[0081] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0082] As described above with reference to the accompanying drawings, the emotional image editing method and system based on the multi-agent collaborative framework proposed in the present invention are described in an exemplary manner. However, those skilled in the art should understand that various improvements can be made to the emotional image editing method and system based on the multi-agent collaborative framework proposed in the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. An emotional image editing method based on a multi-agent collaborative framework, characterized in that: It includes the pre-creation phase and the optimization phase; In the pre-creation stage, a preliminary editing scheme is generated by a planning agent based on the acquired source image and target emotion; the preliminary editing scheme is evaluated by an evaluation agent, and the preliminary editing operation in the preliminary editing scheme is performed by an editing agent to obtain a preliminary edited image; In the optimization stage, the preliminary edited image is evaluated by an evaluation agent. If the preliminary edited image does not meet the target emotion, an optimization editing instruction is generated, and the preliminary edited image is adjusted according to the optimization editing instruction by an editing agent.

2. The emotional image editing method based on a multi-agent collaborative framework as claimed in claim 1, characterized in that: The pre-creation phase includes: S111: analyzing the emotion and semantics of the source image by the planning agent to obtain the emotion information and semantic information of the source image; S112: Based on the target emotion, retrieve emotion editing elements related to the target emotion from an emotion editing element database through a preset emotion factor knowledge retriever; S113: generating a preliminary editing scheme of the emotional image according to the emotional editing elements, wherein the preliminary editing includes specific editing operations and adjustment sequence; S114: Evaluating the preliminary editing plan through an evaluation agent, and optimizing the preliminary editing plan according to the evaluation result to form an optimized editing plan; S115: The editing agent performs the image editing task according to the optimized editing scheme to generate a preliminary edited image.

3. The emotional image editing method based on a multi-agent collaborative framework as claimed in claim 2, characterized in that: The optimization phase includes: S121: Evaluating the preliminary editing result through an evaluation agent to determine whether the preliminary editing result effectively conveys the target emotion; wherein, if the preliminary editing result fails to achieve the expected effect, the evaluation agent provides an optimization editing instruction for the preliminary editing result; S122: The editing agent iteratively updates the preliminary editing result according to the optimized editing instruction to obtain an optimized edited image.

4. The emotional image editing method based on the multi-agent collaborative framework according to claim 3, wherein: The planning agent determines the specific steps of each editing element and editing operation in the preliminary editing plan through a three-layer decision tree structure; wherein the three-layer decision tree takes the target emotion as the root node, identifies the appropriate editing elements as branch nodes, and assigns the corresponding editing methods as leaf nodes.

5. The emotional image editing method based on a multi-agent collaborative framework according to claim 2, wherein: The emotional factor knowledge retriever uses a method based on retrieval enhancement generation to retrieve image elements related to the target emotion from a preset emotional data set to generate the preliminary editing plan.

6. The emotional image editing method based on a multi-agent collaborative framework according to claim 2, wherein: In the pre-creation stage, the editing agent implements the contents of all editing instructions in the optimized editing scheme on the source image step by step in an iterative manner according to the optimized editing scheme; wherein an image editing algorithm tool library is integrated in the editing agent, and in the image editing algorithm tool library, different editing methods correspond to different image editing algorithms.

7. The emotional image editing method based on a multi-agent collaborative framework according to claim 1 is characterized in that: For the editing results of the pre-creation phase, the evaluation, adjustment processing and re-editing operations of the optimization phase are repeated until the evaluation agent obtains an evaluation result that satisfies the expression of the target emotion.

8. A multi-agent collaboration framework, used to implement the emotional image editing method according to any one of claims 1 to 7, characterized in that: include: a planning agent for receiving a target emotion and a source image and generating a preliminary editing scheme according to the source image and the target emotion; An editing agent, configured to edit the source image according to the preliminary editing scheme so that the edited image meets the requirements of the target emotion in terms of vision and emotion; An evaluation agent is used to evaluate the preliminary editing plan and the editing results edited according to the preliminary editing plan, and to put forward optimization suggestions based on the evaluation results to form an optimized editing plan; Also, it is used to evaluate the edited images, determine whether they effectively convey the target emotion, and provide optimization suggestions for editing results that do not meet the standards.

9. An emotional image editing system based on a multi-agent collaborative framework, using the emotional image editing method based on a multi-agent collaborative framework as described in any one of claims 1 to 7 to perform emotional image editing, comprising a pre-creation unit and an optimization unit; wherein, The pre-creation unit is used to generate a preliminary editing scheme based on the acquired source image and target emotion through a planning agent; evaluate the preliminary editing scheme through an evaluation agent, and perform preliminary editing operations in the preliminary editing scheme through an editing agent to obtain a preliminary edited image; The optimization unit is used to evaluate the preliminary edited image through an evaluation agent, and if the preliminary edited image does not meet the target emotion, optimize the editing instructions, and adjust the preliminary edited image according to the optimized editing instructions through the editing agent.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the steps in the emotional image editing method based on a multi-agent collaborative framework as described in any one of claims 1-7.

Images