Image interactive editing method and device, electronic equipment and readable storage medium

Abstract

The invention provides an image interactive editing method and device, electronic equipment and a readable storage medium. The method comprises the steps of obtaining at least one round of interactive dialogue data input by a user; inputting the at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and outputting a plurality of target drawing skills and at least one drawing parameter corresponding to each target drawing skill; and calling at least one target drawing pipeline based on the plurality of target drawing skills and the corresponding at least one drawing parameter, and editing and generating a target drawing image through the target drawing pipeline. Inputting the obtained interactive dialogue data of the user into a pre-trained dialogue parameter extraction model, directly and automatically calling a target drawing pipeline according to the output multiple target drawing skills and at least one drawing parameter corresponding to each target drawing skill, and extracting the interactive dialogue data of the user on the basis of the at least one drawing parameter through the target drawing pipeline. And editing and generating the target drawing image so as to improve the efficiency and accuracy of editing and generating the drawing image.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an interactive editing method, device, electronic device and readable storage medium for images. Background Art

[0002] In the embodiments of the present application, with the development of artificial intelligence (AI) technology, different fields have produced technologies that use AI technology to perform interactions to varying degrees. In the field of image editing, AI painting tools provide different image editing methods, such as partial redrawing, beauty, high-definition magnification, etc. There are thousands of image editing methods in the community.

[0003] In the related art, if an image needs to be edited, different methods need to be called for editing. Even the same type of image editing may use different editing methods due to differences in image editing requirements. In the image editing process, one editing method is a drawing pipeline. When performing image editing, the user needs to professionally select the required drawing pipeline, which makes the user's image editing process too cumbersome. At the same time, due to the different selected drawing pipelines, the image editing results may deviate from the user's expectations, resulting in low efficiency and accuracy in editing and generating painting images. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide an interactive editing method, device, electronic device and readable storage medium for images, input the acquired user's interactive dialogue data into a pre-trained dialogue parameter extraction model, analyze the user's image editing needs, output multiple target painting skills and at least one painting parameter corresponding to each target painting skill, and automatically call the target painting pipeline according to the target painting skill. Through the target painting pipeline, directly based on at least one painting parameter, the target painting image is edited and generated, which helps to improve the efficiency and accuracy of editing and generating painting images.

[0005] In a first aspect, an embodiment of the present application provides an interactive editing method for an image, the interactive editing method comprising:

[0006] Obtain at least one round of interactive dialogue data input by the user;

[0007] Inputting at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and outputting a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill;

[0008] Based on the multiple target painting skills and the corresponding at least one painting parameter, at least one target painting pipeline is called, and a target painting image is edited and generated through the target painting pipeline.

[0009] In a second aspect, an embodiment of the present application further provides an interactive editing device for an image, the interactive editing device comprising:

[0010] A data acquisition module, used to acquire at least one round of interactive dialogue data input by the user;

[0011] A parameter output module, used to input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and output a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill;

[0012] The image generation module is used to call at least one target painting pipeline based on the multiple target painting skills and the corresponding at least one painting parameter, and edit and generate a target painting image through the target painting pipeline.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the interactive image editing method as described in any one of the first aspects.

[0014] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the interactive image editing method as described in any one of the first aspects are executed.

[0015] The interactive editing method, device, electronic device and readable storage medium of an image provided by the embodiment of the present application obtain at least one round of interactive dialogue data input by a user; input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, output multiple target painting skills and at least one painting parameter corresponding to each target painting skill; based on the multiple target painting skills and the corresponding at least one painting parameter, call at least one target painting pipeline, and edit and generate a target painting image through the target painting pipeline. In this way, the obtained interactive dialogue data of the user is input into the pre-trained dialogue parameter extraction model, the user's image editing needs are analyzed, multiple target painting skills and at least one painting parameter corresponding to each target painting skill are output, and the target painting pipeline is automatically called according to the target painting skill, and the target painting image is edited and generated directly based on at least one painting parameter through the target painting pipeline, which helps to improve the efficiency and accuracy of editing and generating painting images.

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of an interactive editing method of an image provided by an embodiment of the present application;

[0019] Figure 2 A flow chart for generating interactive dialogue data provided in an embodiment of the present application;

[0020] Figure 3 A flow chart of the dialogue parameter extraction model training provided in the embodiment of the present application;

[0021] Figure 4 A similarity score calculation flow chart provided in an embodiment of the present application;

[0022] Figure 5 One of the structural schematic diagrams of an interactive editing device for an image provided in an embodiment of the present application;

[0023] Figure 6 The second structural diagram of an interactive editing device for an image provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0026] In the embodiments of the present application, with the development of artificial intelligence (AI) technology, different fields have produced technologies that use AI technology to perform interactions to varying degrees. In the field of image editing, AI painting tools provide different image editing methods, such as partial redrawing, beauty, high-definition magnification, etc. There are thousands of image editing methods in the community.

[0027] In the related art, if an image needs to be edited, different methods need to be called for editing. Even the same type of image editing may use different editing methods due to differences in image editing requirements. In the image editing process, one editing method is a drawing pipeline. When performing image editing, the user needs to professionally select the required drawing pipeline, which makes the user's image editing process too cumbersome. At the same time, due to the different selected drawing pipelines, the image editing results may deviate from the user's expectations, resulting in low efficiency and accuracy in editing and generating painting images.

[0028] Based on this, an embodiment of the present application provides an interactive editing method for an image to improve the efficiency and accuracy of editing and generating a painting image.

[0029] See also Figure 1 , Figure 1 This is a flow chart of an interactive editing method of an image provided by an embodiment of the present application. Figure 1 As shown in , the interactive editing method of an image provided by an embodiment of the present application includes:

[0030] S101: Obtain at least one round of interactive dialogue data input by a user.

[0031] S102: Input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and output a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill.

[0032] S103: Based on the multiple target painting skills and the corresponding at least one painting parameter, call at least one target painting pipeline, and edit and generate a target painting image through the target painting pipeline.

[0033] The interactive image editing method provided in the embodiment of the present application inputs the acquired user's interactive dialogue data into a pre-trained dialogue parameter extraction model, analyzes the user's image editing needs, outputs multiple target painting skills and at least one painting parameter corresponding to each target painting skill, and automatically calls the target painting pipeline according to the target painting skills. The target painting image is edited and generated directly based on the at least one painting parameter through the target painting pipeline, which helps to improve the efficiency and accuracy of editing and generating painting images.

[0034] The following is a description of the exemplary steps of the present application embodiment:

[0035] S101: Obtain at least one round of interactive dialogue data input by a user.

[0036] In the embodiments of the present application, with the development of artificial intelligence (AI) technology, different fields have produced technologies that use AI technology to perform interactions to varying degrees. In the field of image editing, AI painting tools provide different image editing methods, such as partial redrawing, beauty, high-definition magnification, etc. There are thousands of image editing methods in the community.

[0037] In the related art, if an image needs to be edited, different methods need to be called for editing. Even the same type of image editing may use different editing methods due to differences in image editing requirements. In the image editing process, an editing method is a drawing pipeline. When performing image editing, the user needs to professionally select the required drawing pipeline, which makes the user's image editing process too cumbersome. At the same time, due to the different selected drawing pipelines, the image editing results may deviate from the user's expectations.

[0038] Based on this, the present application considers analyzing the user's interactive dialogue data to accurately analyze the user's painting needs and painting purposes, and then obtain accurate image editing results.

[0039] In a possible implementation, when obtaining the interactive dialogue data input by the user, the text dialogue data input by the user may be directly obtained, or the voice dialogue data input by the user may be obtained, and then the voice dialogue data is converted into text dialogue data, and then analyzed based on the obtained interactive dialogue data.

[0040] Specifically, the step of “obtaining at least one round of interactive dialogue data input by the user” includes:

[0041] a1: If the interactive dialogue data input by the user is text information, generate at least one round of interactive dialogue data based on the text information.

[0042] a2: If the interactive dialogue data input by the user is voice information, convert the voice information input by the user into text information to generate at least one round of interactive dialogue data.

[0043] In an embodiment of the present application, when the interactive conversation data input by the user is text information, when the terminal on which the user performs image editing is a mobile terminal, the user can input the text information in the text input interface displayed in the graphical user interface by touching the graphical user interface of the mobile terminal; when the terminal on which the user performs image editing is a PC, the user can input the text conversation data through an external input device (for example, a keyboard); when the interactive conversation data input by the user is voice information, the user's voice information can be collected through a microphone on the mobile terminal or the PC.

[0044] Furthermore, if it is determined that at least one round of interactive dialogue data from the user has been received, the obtained at least one round of interactive dialogue data can be input into a pre-trained dialogue parameter extraction model. Through processing by the dialogue parameter extraction model, multiple target painting skills contained in the at least one round of interactive dialogue data from the user and at least one painting parameter corresponding to each target painting skill can be obtained.

[0045] Here, providing different methods for acquiring user interaction dialogue data can enrich the user's dialogue input method, thereby improving the efficiency of human-computer interaction.

[0046] S102: Input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and output a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill.

[0047] In an embodiment of the present application, after at least one round of interactive dialogue data is input into a pre-trained dialogue parameter extraction model, the dialogue parameter extraction model will analyze whether there is dialogue data related to image editing in the interactive dialogue data of the current user. If so, the corresponding target painting skill and at least one painting parameter corresponding to the target painting skill are output based on the interactive dialogue data.

[0048] The target painting skills include at least one of the following:

[0049] Drawing skills, image magnification skills, image style conversion skills, image partial redrawing skills, image optimization skills, image background update skills.

[0050] In a possible implementation, for different painting pipelines, the required input parameters may differ to a certain extent. Therefore, when analyzing at least one painting parameter of the target painting skill, the dialogue parameter extraction model needs to refer to the analyzed input parameters of the painting pipeline corresponding to the target painting skill, and finally determine at least one painting parameter of the target painting skill.

[0051] Exemplarily, if the user's interactive dialogue data is a single-round dialogue data, including "replace target object 1 in the image with target object 2", at this time, it is analyzed that the current target painting skill is "image local redrawing skill", and the parameters of the painting pipeline corresponding to the "image local redrawing skill" are vocabulary parameters. Then, the painting parameters obtained at this time are "replace target object 1 with target object 2"; if the user's interactive dialogue data is a single-round dialogue data, including "draw a boat moving in the water", at this time, it is analyzed that the current target painting skill is "drawing skill", and the parameters of the painting pipeline corresponding to the "drawing skill" are text parameters. Then, the painting parameters obtained at this time are "draw a boat moving in the water".

[0052] In a possible implementation, if the dialogue parameter extraction model analyzes that there is no dialogue data related to image editing in the interactive dialogue data of the current user, it is determined that the interactive dialogue data of the current user does not mention the need for image editing. At this time, the dialogue parameter extraction model will no longer output the target painting skills and painting parameters required for image editing, but will return a preset string to interact with the user through the preset string.

[0053] Among them, the preset character string can be pre-set by the model designer, and can be a greeting phrase used to interact with the user, such as "Hello"; it can also be an interactive phrase to guide the user to edit the image, such as "Please enter the image editing requirements" and so on.

[0054] In a possible implementation, in order to ensure the accuracy of the dialogue parameter extraction model's analysis of the user's interactive dialogue data, and thereby improve the matching degree between the generated target painting image and the user's image editing requirements, the training and adjustment process of the dialogue parameter extraction model is relatively important. The training process of the dialogue parameter extraction model will be specifically described below.

[0055] Specifically, the dialogue parameter extraction model is trained through the following steps:

[0056] b1: Obtain multiple sets of generated sample interactive dialogue data including dialogue data of different rounds and sample images corresponding to each round of dialogue data.

[0057] b2: Input each group of sample interactive dialogue data into the constructed multimodal large language model, and output sample painting skills and sample painting parameters.

[0058] b3: Generate training target images based on sample painting skills and sample painting parameters.

[0059] b4: Detect whether there is an image where the training target image is inconsistent with the corresponding sample image; if so, adjust the model parameters of the multimodal large language model until each training target image obtained is consistent with the corresponding sample image, determine that the training of the multimodal large language model is completed, and obtain the dialogue parameter extraction model.

[0060] In a possible implementation, multiple groups of sample interactive dialogue data are generated by the following steps:

[0061] c1: Obtain the painting models and painting pipelines used in the historical image editing process, and determine the input parameters and functional description information corresponding to each painting model and painting pipeline.

[0062] c2: Generate multiple sets of sample interactive dialogue data based on the input parameters and functional description information corresponding to each painting model and painting pipeline.

[0063] In an embodiment of the present application, a preset historical time period (a historical week or month, etc.) may be obtained, a painting model for image editing and a painting pipeline consisting of multiple painting models may be collected, and input parameters and functional description information corresponding to each painting model and painting pipeline may be determined; and then, multiple groups of sample interactive dialogue data may be generated according to the input parameters and functional description information corresponding to the corresponding painting models and painting pipelines.

[0064] In a possible implementation, in order to ensure that the generated sample interactive dialogue data is targeted, multiple candidate painting images may be determined through multiple prompt information obtained in advance, and corresponding sample interactive dialogue data may be generated based on corresponding image-text pairs.

[0065] Specifically, the step of "generating multiple sets of sample interactive dialogue data based on the input parameters and functional description information corresponding to each painting model and painting pipeline" includes:

[0066] d1: Generate multiple candidate painting images for each input parameter and function description information obtained.

[0067] d2: For each candidate painting image, generate a plurality of editing instructions for the candidate painting image.

[0068] d3: For each candidate painting image, a plurality of editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain a plurality of groups of sample interactive dialogue data corresponding to the candidate painting image.

[0069] In an embodiment of the present application, multiple prompt words may be generated through AI technology based on each input parameter and function description information obtained, or multiple prompt words may be directly generated; or, multiple prompt words may be extracted based on historical interactive dialogue data, and then multiple candidate painting images may be generated based on the prompt words through a text-based graph model.

[0070] In a possible implementation, each set of sample interactive dialogue data is generated based on the painting image as a division basis, and multiple painting skills in image editing are referred to. For each candidate painting image, multiple editing instructions for the candidate painting image are generated according to the prompt words corresponding to the candidate painting image.

[0071] Here, for each candidate painting image, the number of editing instructions generated is consistent with the number of painting skills.

[0072] Furthermore, for each candidate painting image, multiple editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain multiple groups of sample interactive dialogue data corresponding to the candidate painting image.

[0073] Among them, according to the user's dialogue interaction habits, the multiple groups of sample interactive dialogue data include single-round dialogue data, multi-round dialogue data greater than a preset dialogue round, and multi-round dialogue data less than a preset dialogue round.

[0074] Exemplarily, the preset number of conversation rounds may be three rounds. According to an analysis of the user's conversation interaction habits, generally a single-round conversation interaction has the largest number of interactions. The higher the conversation round, the less corresponding interactive conversation data. Therefore, when randomly combining conversation data, the preset conversation round ratio may be the number of interactive conversation data in a single-round conversation: the number of interactive conversation data less than three rounds but greater than a single-round conversation: the number of interactive conversation data less than three rounds = 6:3:1.

[0075] In a possible implementation, if there are K painting skills and N candidate painting images are determined, the number of sample interactive dialogue data that can be obtained may be n*k.

[0076] For example, see Figure 2 , Figure 2 The interactive dialogue data generation flow chart provided in the embodiment of the present application is as follows: Figure 2As shown, first, prompt words are generated through intelligent technology; then, based on the generated prompt words, multiple image-text pairs are obtained through image-text conversion; the image-text pairs are processed in combination with the input and output of the intention, and multiple instructions are randomly generated based on intelligent technology to obtain interactive dialogue data.

[0077] Furthermore, after determining multiple groups of sample interactive dialogue data, different rounds can be input into the constructed multimodal large language model, and then the sample painting skills and sample painting parameters can be output according to the multimodal large language model. The convergence of the current multimodal large language model can be determined by setting a loss function. When it is determined that the loss function value is less than the preset loss function value, it is determined that the training of the multimodal large language model is completed, and then the dialogue parameter extraction model is obtained.

[0078] In one possible implementation, the loss function may be a loss function based on a distance metric, such as Euclidean distance, Hamming distance, etc.; it may also be a loss function of probability distribution difference, such as a cross entropy loss function; it may also be other types of loss functions, such as KL divergence loss, binary cross entropy loss, etc.

[0079] In another possible implementation, it can also be sample interactive dialogue data that has been manually verified. At the same time, the sample image corresponding to each round of dialogue data is determined, and after the sample interactive dialogue data is input into the constructed multimodal large language model, the sample painting skills and sample painting parameters are obtained, and then the corresponding training target image is generated according to the sample painting skills and sample painting parameters, and the obtained training target image is compared with the manually verified sample image. If there is a situation where the training target image is inconsistent with the manually verified sample image, it is determined that the multimodal large language model obtained by the current training still has a certain deviation. At this time, it is necessary to adjust the model parameters of the multimodal large language model again until it is determined that the obtained training target image is consistent with the corresponding sample image. Or the proportion of images that are inconsistent with the training target image and the corresponding sample image is less than the preset ratio threshold. At this time, it is determined that the training of the multimodal large language model is completed and the dialogue parameter extraction model is obtained.

[0080] In one possible implementation, the multimodal large language model can be trained based on LoRA (Low-Rank Adaptation), which can create a variety of AI painting styles, while reducing the cost of AI painting and improving the diversity and flexibility of AI painting.

[0081] For example, see Figure 3 , Figure 3 The training flow chart of the dialogue parameter extraction model provided in the embodiment of the present application is as follows: Figure 3As shown, based on the single-round interactive dialogue data, 2 to 3 rounds of interactive dialogue data, and more than 3 rounds of interactive dialogue data, a multimodal large language model is combined to perform LoRA-based training to obtain a trained dialogue parameter extraction model.

[0082] In a possible implementation, the dialogue parameter extraction model may be further optimized and updated by evaluating the dialogue parameter extraction model, thereby further improving the parameter extraction accuracy of the dialogue parameter extraction model.

[0083] Specifically, the interactive editing method further includes:

[0084] e1: For each set of sample interactive dialogue data, calculate the similarity score between the training target image generated by the sample interactive dialogue data and the sample interactive dialogue data.

[0085] e2: Determine the number of dialogue data having a corresponding similarity score less than a preset similarity score threshold in the multiple groups of sample interactive dialogue data.

[0086] e3: If the number of dialogue data with a similarity score less than a preset similarity score threshold is greater than a preset number threshold, the model parameters of the dialogue parameter extraction model are updated and optimized.

[0087] In a possible implementation, after the dialogue parameter extraction model outputs sample painting skills and sample painting parameters based on the input sample interactive dialogue data, and obtains the training target image based on the sample painting skills and the sample painting parameters, a similarity score between the sample interactive dialogue data and the training target image is calculated. The higher the similarity score, the higher the correlation between the obtained training target image and the sample interactive data, and the higher the parameter extraction accuracy of the dialogue parameter extraction model. After the similarity score calculation for each group of sample interactive dialogue data is completed, the number of dialogue data having a corresponding similarity score less than a preset similarity score threshold among multiple similarity scores is determined. If it is determined that the number of dialogue data having a similarity score less than the preset similarity score threshold is greater than the preset number threshold, it is determined that there is a difference in the correlation between the obtained training target image and the input interactive dialogue data, and the parameter extraction accuracy of the dialogue parameter extraction model is low. Therefore, it is necessary to optimize and adjust the model parameters of the dialogue parameter extraction model to further improve the parameter extraction accuracy of the dialogue parameter extraction model.

[0088] In a possible implementation, in order to improve the accuracy of calculating the similarity score between the training target image and the sample interactive dialogue data, it is necessary to adopt different interactive dialogue data calculation methods for different rounds.

[0089] Specifically, the step of “for each set of sample interactive dialogue data, calculating the similarity score between the training target image generated by the sample interactive dialogue data and the sample interactive dialogue data” includes:

[0090] f1: For each set of sample interactive dialogue data, if the current is the initial dialogue round in the sample interactive dialogue data, the similarity score is calculated based on the sample interactive dialogue data and the obtained training target image.

[0091] f2: For each set of sample interactive dialogue data, if the current dialogue is not the initial dialogue round in the sample interactive dialogue data, multiple dialogue rounds of interactive dialogue data are determined based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round.

[0092] f3: Calculate the similarity score based on the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

[0093] In a possible implementation, if it is determined that the current sample interactive dialogue data is an initial dialogue round, the similarity score between the sample interactive dialogue data of the initial dialogue round and the obtained training target image is directly calculated.

[0094] In another possible implementation, if it is determined that the current sample interactive dialogue data is not the initial dialogue round, it is necessary to summarize multiple rounds of interactive dialogue data based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round, and obtain a similarity score by calculating the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

[0095] For example, see Figure 4 , Figure 4 The similarity score calculation flow chart provided in the embodiment of the present application is as follows: Figure 4 As shown in , if the current sample interactive dialogue data is the dialogue data of the initial dialogue round, a first similarity score is calculated based on the first dialogue data of the initial dialogue round and the first training target image obtained in the initial round; if the current sample interactive dialogue data is the dialogue data of the second dialogue round, a second similarity score is calculated based on the first dialogue data of the initial dialogue round, the second dialogue data of the second round, and the second training target image obtained in the second round; if the current sample interactive dialogue data is the dialogue data of the third dialogue round, a third similarity score is calculated based on the first dialogue data of the initial dialogue round, the second dialogue data of the second round, the third dialogue data of the third round, and the third training target image obtained in the third round; and then the final target similarity score is obtained.

[0096] In one possible implementation, a multimodal (text and image) pre-training model (Contrastive Language-Image Pre-training, CLIP) may be used to calculate a similarity score between sample interactive dialogue data and a training target image.

[0097] Furthermore, after determining multiple target painting skills and at least one painting parameter corresponding to each target painting skill through the dialogue parameter extraction model, the corresponding target painting pipeline can be called according to the obtained multiple target painting skills and at least one painting parameter corresponding to each target painting skill, thereby generating an edited target painting image to meet the user's image editing needs.

[0098] S103: Based on the multiple target painting skills and the corresponding at least one painting parameter, call at least one target painting pipeline, and edit and generate a target painting image through the target painting pipeline.

[0099] In a possible implementation, the parameters and number of parameters required by different painting pipelines are certain. When generating a target painting image through the painting pipeline, the parameters need to be accurate and sufficient. Therefore, when generating a target painting image, it is necessary to detect whether the currently acquired painting parameters are complete. If there are omissions, it is necessary to interact with the user to guide the user to re-enter the missing painting parameters.

[0100] Specifically, the step of "generating a target painting image through editing of the target painting pipeline" includes:

[0101] g1: Check whether the current at least one drawing parameter contains all the parameters required by the target drawing pipeline.

[0102] g2: If the current at least one drawing parameter includes all parameters required by the target drawing pipeline, based on the at least one drawing parameter, edit and generate the target drawing image through the target drawing pipeline.

[0103] In a possible implementation, if it is determined that the at least one drawing parameter parsed contains all parameters required by the target drawing pipeline, the target drawing image is directly generated by editing through the target drawing pipeline according to the at least one drawing parameter.

[0104] For example, if the target painting skill is "update image background skill", the corresponding painting parameters required by the target painting pipeline are "image, background, background update target", and through dialogue parameter extraction, at least one painting parameter obtained includes image, background, background update target, which contains all the parameters required by the target painting pipeline, and the target painting image can be directly generated based on the painting parameters.

[0105] In another possible implementation, the step of “generating a target painting image by editing the target painting pipeline” further includes:

[0106] h1: if the current at least one drawing parameter does not include all the parameters required by the target drawing pipeline, generate drawing parameter question information, and prompt the user to input the missing supplementary drawing parameters based on the drawing parameter question information.

[0107] h2: When it is determined that the at least one drawing parameter and the at least one supplementary drawing parameter input by the user include all parameters required by the target drawing pipeline, a target drawing image is edited and generated through the target drawing pipeline.

[0108] In an embodiment of the present application, for the above example, if the painting parameters required by the target painting pipeline are "image, background, background update target", at least one painting parameter obtained through dialogue parameter extraction includes image and background, but does not include the required background update target. At this time, it is determined that the painting parameters of the target painting pipeline are missing, and the user needs to be prompted to supplement the parameters.

[0109] In a possible implementation, the generated painting parameter question information may include the currently required painting parameters, for example, an imperative sentence "Please re-enter XX parameters" or a question "Do you need to define XX parameters?". The user needs to be clearly prompted with the painting parameters that are currently required to be supplemented in the painting parameter question information. The supplemented painting parameters are obtained by inputting the acquired user supplementary interactive dialogue data into the dialogue parameter extraction model, and then the target painting pipeline is called to combine the painting parameters and the supplemented parameters to generate a target painting image.

[0110] In this way, by interactively supplementing parameters through asking questions, the user's image editing needs can be understood more accurately and the normal operation of the target painting pipeline can be ensured, further improving the accuracy of generating the target painting image.

[0111] The interactive editing method of an image provided by the embodiment of the present application obtains at least one round of interactive dialogue data input by a user; inputs at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, outputs multiple target painting skills and at least one painting parameter corresponding to each target painting skill; based on the multiple target painting skills and the corresponding at least one painting parameter, calls at least one target painting pipeline, and edits and generates a target painting image through the target painting pipeline. In this way, the obtained interactive dialogue data of the user is input into the pre-trained dialogue parameter extraction model, analyzes the image editing needs of the user, outputs multiple target painting skills and at least one painting parameter corresponding to each target painting skill, and automatically calls the target painting pipeline according to the target painting skill, and edits and generates the target painting image directly based on at least one painting parameter through the target painting pipeline, which helps to improve the efficiency and accuracy of editing and generating painting images.

[0112] Based on the same inventive concept, an interactive editing device for images corresponding to the interactive editing method for images is also provided in an embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the interactive editing method for images in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0113] See also Figure 5 , Figure 6 , Figure 5 This is one of the structural schematic diagrams of an interactive editing device for an image provided in an embodiment of the present application. Figure 6 This is a second structural diagram of an interactive editing device for an image provided by an embodiment of the present application. Figure 5 As shown in , the interactive editing device 500 includes:

[0114] The data acquisition module 510 is used to acquire at least one round of interactive dialogue data input by the user;

[0115] A parameter output module 520, configured to input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and output a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill;

[0116] The image generation module 530 is used to call at least one target painting pipeline based on the multiple target painting skills and the corresponding at least one painting parameter, and edit and generate a target painting image through the target painting pipeline.

[0117] In a possible implementation, Figure 6 As shown, the interactive editing device 500 further includes a model training module 540, and the model training module 540 is used to train the dialogue parameter extraction model through the following steps:

[0118] Acquire multiple sets of sample interactive dialogue data including different rounds of dialogue data and sample images corresponding to each round of dialogue data;

[0119] Input each group of sample interactive dialogue data into the constructed multimodal large language model, and output sample painting skills and sample painting parameters;

[0120] Generate a training target image based on the sample painting skills and the sample painting parameters;

[0121] Detect whether there are images whose training target images are inconsistent with the corresponding sample images; if so, adjust the model parameters of the multimodal large language model until each training target image obtained is consistent with the corresponding sample image, determine that the training of the multimodal large language model is completed, and obtain the dialogue parameter extraction model.

[0122] In a possible implementation, the model training module 540 is used to generate multiple groups of sample interactive dialogue data through the following steps:

[0123] Obtain the painting models and painting pipelines used in the historical image editing process, and determine the input parameters and functional description information corresponding to each painting model and painting pipeline;

[0124] Based on the input parameters and functional description information corresponding to each painting model and painting pipeline, multiple sets of sample interactive dialogue data are generated.

[0125] In a possible implementation, when the model training module 540 is used to generate multiple sets of sample interactive dialogue data based on input parameters and functional description information corresponding to each painting model and painting pipeline, the model training module 540 is used to:

[0126] For each input parameter and function description information obtained, multiple candidate painting images are generated;

[0127] For each candidate painting image, generate a plurality of editing instructions for the candidate painting image;

[0128] For each candidate painting image, multiple editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain multiple groups of sample interactive dialogue data corresponding to the candidate painting image.

[0129] In a possible implementation, the multiple groups of sample interactive dialogue data include single-round dialogue data, multi-round dialogue data greater than a preset dialogue round, and multi-round dialogue data less than a preset dialogue round.

[0130] In a possible implementation, Figure 6As shown, the interactive editing device 500 further includes a model optimization module 550, and the model optimization module 550 is used to:

[0131] For each set of sample interactive dialogue data, calculating a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data;

[0132] Determine the number of conversation data having a corresponding similarity score less than a preset similarity score threshold in the plurality of groups of sample interactive conversation data;

[0133] If the number of dialogue data having a similarity score less than a preset similarity score threshold is greater than a preset number threshold, the model parameters of the dialogue parameter extraction model are updated and optimized.

[0134] In a possible implementation, when the model optimization module 550 is used to calculate, for each set of sample interactive dialogue data, a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data, the model optimization module 550 is used to:

[0135] For each set of sample interactive dialogue data, if the current dialogue is an initial dialogue round in the sample interactive dialogue data, the similarity score is calculated based on the sample interactive dialogue data and the obtained training target image;

[0136] For each set of sample interactive dialogue data, if the current dialogue is not an initial dialogue round in the sample interactive dialogue data, determine multiple dialogue rounds of interactive dialogue data based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round;

[0137] The similarity score is calculated based on the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

[0138] In a possible implementation, when the image generation module 530 is used to edit and generate a target painting image through the target painting pipeline, the image generation module 530 is used to:

[0139] Detecting whether the current at least one drawing parameter includes all parameters required by the target drawing pipeline;

[0140] If the current at least one drawing parameter includes all parameters required by the target drawing pipeline, the target drawing image is edited and generated through the target drawing pipeline based on the at least one drawing parameter.

[0141] In a possible implementation manner, when the image generation module 530 is used to edit and generate the target painting image through the target painting pipeline, the image generation module 530 is further used to:

[0142] If the current at least one drawing parameter does not include all the parameters required by the target drawing pipeline, generating drawing parameter question information, and prompting the user to input the missing supplementary drawing parameters based on the drawing parameter question information;

[0143] When it is determined that the at least one drawing parameter and the at least one supplementary drawing parameter input by the user include all parameters required by the target drawing pipeline, the target drawing image is edited and generated by the target drawing pipeline.

[0144] In a possible implementation, when the data acquisition module 510 is used to acquire at least one round of interactive dialogue data input by a user, the data acquisition module 510 is used to:

[0145] If the interactive dialogue data input by the user is text information, generating at least one round of interactive dialogue data based on the text information;

[0146] If the interactive dialogue data input by the user is voice information, at least one round of interactive dialogue data is generated after the voice information input by the user is converted into text information.

[0147] In a possible implementation, the target painting skill includes at least one of the following:

[0148] Drawing skills, image magnification skills, image style conversion skills, image partial redrawing skills, image optimization skills, image background update skills.

[0149] The interactive editing device for images provided in the embodiment of the present application obtains at least one round of interactive dialogue data input by the user; inputs at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, outputs multiple target painting skills and at least one painting parameter corresponding to each target painting skill; based on the multiple target painting skills and the corresponding at least one painting parameter, calls at least one target painting pipeline, and edits and generates a target painting image through the target painting pipeline. In this way, the obtained interactive dialogue data of the user is input into the pre-trained dialogue parameter extraction model, analyzes the image editing needs of the user, outputs multiple target painting skills and at least one painting parameter corresponding to each target painting skill, and automatically calls the target painting pipeline according to the target painting skill, and directly edits and generates the target painting image based on at least one painting parameter through the target painting pipeline, which helps to improve the efficiency and accuracy of editing and generating painting images.

[0150] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7As shown in , the electronic device 700 includes a processor 710 , a memory 720 and a bus 730 .

[0151] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730, so that the processor 710 executes the following instructions when running:

[0152] Obtain at least one round of interactive dialogue data input by the user;

[0153] Inputting at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and outputting a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill;

[0154] Based on the multiple target painting skills and the corresponding at least one painting parameter, at least one target painting pipeline is called, and a target painting image is edited and generated through the target painting pipeline.

[0155] In a possible implementation, in the instructions executed by the processor 710, the dialogue parameter extraction model is trained through the following steps:

[0156] Acquire multiple sets of sample interactive dialogue data including different rounds of dialogue data and sample images corresponding to each round of dialogue data;

[0157] Input each group of sample interactive dialogue data into the constructed multimodal large language model, and output sample painting skills and sample painting parameters;

[0158] Generate a training target image based on the sample painting skills and the sample painting parameters;

[0159] Detect whether there are images whose training target images are inconsistent with the corresponding sample images; if so, adjust the model parameters of the multimodal large language model until each training target image obtained is consistent with the corresponding sample image, determine that the training of the multimodal large language model is completed, and obtain the dialogue parameter extraction model.

[0160] In a possible implementation manner, in the instructions executed by the processor 710, multiple groups of sample interactive dialogue data are generated by the following steps:

[0161] Obtain the painting models and painting pipelines used in the historical image editing process, and determine the input parameters and functional description information corresponding to each painting model and painting pipeline;

[0162] Based on the input parameters and functional description information corresponding to each painting model and painting pipeline, multiple sets of sample interactive dialogue data are generated.

[0163] In a possible implementation, the instructions executed by the processor 710, wherein the generating of multiple sets of sample interactive dialogue data based on the input parameters and functional description information corresponding to each drawing model and drawing pipeline includes:

[0164] For each input parameter and function description information obtained, multiple candidate painting images are generated;

[0165] For each candidate painting image, generate a plurality of editing instructions for the candidate painting image;

[0166] For each candidate painting image, multiple editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain multiple groups of sample interactive dialogue data corresponding to the candidate painting image.

[0167] In a possible implementation, the multiple groups of sample interactive dialogue data include single-round dialogue data, multi-round dialogue data greater than a preset dialogue round, and multi-round dialogue data less than a preset dialogue round.

[0168] In a possible implementation manner, the instructions executed by the processor 710 further include:

[0169] For each set of sample interactive dialogue data, calculating a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data;

[0170] Determine the number of conversation data having a corresponding similarity score less than a preset similarity score threshold in the plurality of groups of sample interactive conversation data;

[0171] If the number of dialogue data having a similarity score less than a preset similarity score threshold is greater than a preset number threshold, the model parameters of the dialogue parameter extraction model are updated and optimized.

[0172] In a possible implementation manner, in the instructions executed by the processor 710, calculating, for each set of sample interactive dialogue data, a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data includes:

[0173] For each set of sample interactive dialogue data, if the current dialogue is an initial dialogue round in the sample interactive dialogue data, the similarity score is calculated based on the sample interactive dialogue data and the obtained training target image;

[0174] For each set of sample interactive dialogue data, if the current dialogue is not an initial dialogue round in the sample interactive dialogue data, determine multiple dialogue rounds of interactive dialogue data based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round;

[0175] The similarity score is calculated based on the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

[0176] In a possible implementation manner, in the instructions executed by the processor 710, the step of editing and generating a target painting image through the target painting pipeline includes:

[0177] Detecting whether the current at least one drawing parameter includes all parameters required by the target drawing pipeline;

[0178] If the current at least one drawing parameter includes all parameters required by the target drawing pipeline, the target drawing image is edited and generated through the target drawing pipeline based on the at least one drawing parameter.

[0179] In a possible implementation manner, in the instructions executed by the processor 710, the step of editing and generating a target painting image through the target painting pipeline further includes:

[0180] If the current at least one drawing parameter does not include all the parameters required by the target drawing pipeline, generating drawing parameter question information, and prompting the user to input the missing supplementary drawing parameters based on the drawing parameter question information;

[0181] When it is determined that the at least one drawing parameter and the at least one supplementary drawing parameter input by the user include all parameters required by the target drawing pipeline, the target drawing image is edited and generated by the target drawing pipeline.

[0182] In a possible implementation manner, in the instructions executed by the processor 710, the step of obtaining at least one round of interactive dialogue data input by the user includes:

[0183] If the interactive dialogue data input by the user is text information, generating at least one round of interactive dialogue data based on the text information;

[0184] If the interactive dialogue data input by the user is voice information, at least one round of interactive dialogue data is generated after the voice information input by the user is converted into text information.

[0185] In a possible implementation manner, the target painting skill includes at least one of the following:

[0186] Drawing skills, image magnification skills, image style conversion skills, image partial redrawing skills, image optimization skills, image background update skills.

[0187] Through the above method, the obtained interactive dialogue data of the user is input into the pre-trained dialogue parameter extraction model, the user's image editing needs are analyzed, multiple target painting skills and at least one painting parameter corresponding to each target painting skill are output, and the target painting pipeline is automatically called according to the target painting skill. The target painting pipeline directly edits and generates the target painting image based on at least one painting parameter, which helps to improve the efficiency and accuracy of editing and generating painting images. At the same time, providing different methods for obtaining user interactive dialogue data can enrich the user's dialogue input method, thereby improving the efficiency of human-computer interaction; by calculating the similarity score, the model parameters of the dialogue parameter extraction model are optimized and adjusted, and the parameter extraction accuracy of the dialogue parameter extraction model is further improved; different interactive dialogue data calculation methods are used for different rounds, which also improves the accuracy of similarity score calculation. Furthermore, according to the method of asking questions to interactively supplement parameters, the user's image editing needs can be more accurately understood and the normal operation of the target painting pipeline can be guaranteed, further improving the accuracy of target painting image generation.

[0188] The present application also provides a computer-readable storage medium, on which a computer program is stored, so that when the computer program is executed by a processor, the following instructions are executed:

[0189] Obtain at least one round of interactive dialogue data input by the user;

[0190] Inputting at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and outputting a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill;

[0191] Based on the multiple target painting skills and the corresponding at least one painting parameter, at least one target painting pipeline is called, and a target painting image is edited and generated through the target painting pipeline.

[0192] In a possible implementation manner, in the instructions executed by the computer-readable storage medium, the dialogue parameter extraction model is trained by the following steps:

[0193] Acquire multiple sets of sample interactive dialogue data including dialogue data of different rounds and sample images corresponding to each round of dialogue data;

[0194] Input each group of sample interactive dialogue data into the constructed multimodal large language model, and output sample painting skills and sample painting parameters;

[0195] Generate a training target image based on the sample painting skills and the sample painting parameters;

[0196] Detect whether there are images whose training target images are inconsistent with the corresponding sample images; if so, adjust the model parameters of the multimodal large language model until each training target image obtained is consistent with the corresponding sample image, determine that the training of the multimodal large language model is completed, and obtain the dialogue parameter extraction model.

[0197] In a possible implementation, in the instructions executed by the computer-readable storage medium, multiple groups of sample interactive dialogue data are generated by the following steps:

[0198] Obtain the painting models and painting pipelines used in the historical image editing process, and determine the input parameters and functional description information corresponding to each painting model and painting pipeline;

[0199] Based on the input parameters and functional description information corresponding to each painting model and painting pipeline, multiple sets of sample interactive dialogue data are generated.

[0200] In a possible implementation, the instructions executed by the computer-readable storage medium, wherein the steps of generating multiple sets of sample interactive dialogue data based on input parameters and functional description information corresponding to each drawing model and drawing pipeline include:

[0201] For each input parameter and function description information obtained, multiple candidate painting images are generated;

[0202] For each candidate painting image, generate a plurality of editing instructions for the candidate painting image;

[0203] For each candidate painting image, multiple editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain multiple groups of sample interactive dialogue data corresponding to the candidate painting image.

[0204] In a possible implementation, the multiple groups of sample interactive dialogue data include single-round dialogue data, multi-round dialogue data greater than a preset dialogue round, and multi-round dialogue data less than a preset dialogue round.

[0205] In a possible implementation manner, the instructions executed by the computer-readable storage medium further include:

[0206] For each set of sample interactive dialogue data, calculating a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data;

[0207] Determine the number of conversation data having a corresponding similarity score less than a preset similarity score threshold in the plurality of groups of sample interactive conversation data;

[0208] If the number of dialogue data having a similarity score less than a preset similarity score threshold is greater than a preset number threshold, the model parameters of the dialogue parameter extraction model are updated and optimized.

[0209] In a possible implementation manner, in the instructions executed by a computer-readable storage medium, the step of calculating, for each set of sample interactive dialogue data, a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data includes:

[0210] For each set of sample interactive dialogue data, if the current dialogue is an initial dialogue round in the sample interactive dialogue data, the similarity score is calculated based on the sample interactive dialogue data and the obtained training target image;

[0211] For each set of sample interactive dialogue data, if the current dialogue is not an initial dialogue round in the sample interactive dialogue data, determine multiple dialogue rounds of interactive dialogue data based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round;

[0212] The similarity score is calculated based on the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

[0213] In a possible implementation manner, in the instructions executed by the computer-readable storage medium, the step of editing and generating a target painting image through the target painting pipeline includes:

[0214] Detecting whether the current at least one drawing parameter includes all parameters required by the target drawing pipeline;

[0215] If the current at least one drawing parameter includes all parameters required by the target drawing pipeline, the target drawing image is edited and generated through the target drawing pipeline based on the at least one drawing parameter.

[0216] In a possible implementation manner, in the instructions executed by the computer-readable storage medium, the step of editing and generating a target painting image through the target painting pipeline further includes:

[0217] If the current at least one drawing parameter does not include all the parameters required by the target drawing pipeline, generating drawing parameter question information, and prompting the user to input the missing supplementary drawing parameters based on the drawing parameter question information;

[0218] When it is determined that the at least one drawing parameter and the at least one supplementary drawing parameter input by the user include all parameters required by the target drawing pipeline, the target drawing image is edited and generated by the target drawing pipeline.

[0219] In a possible implementation manner, in the instructions executed by the computer-readable storage medium, the step of obtaining at least one round of interactive dialogue data input by the user includes:

[0220] If the interactive dialogue data input by the user is text information, generating at least one round of interactive dialogue data based on the text information;

[0221] If the interactive dialogue data input by the user is voice information, at least one round of interactive dialogue data is generated after the voice information input by the user is converted into text information.

[0222] In a possible implementation manner, the target painting skill includes at least one of the following:

[0223] Drawing skills, image magnification skills, image style conversion skills, image partial redrawing skills, image optimization skills, image background update skills.

[0224] Through the above method, the obtained interactive dialogue data of the user is input into the pre-trained dialogue parameter extraction model, the user's image editing needs are analyzed, multiple target painting skills and at least one painting parameter corresponding to each target painting skill are output, and the target painting pipeline is automatically called according to the target painting skill. The target painting pipeline directly edits and generates the target painting image based on at least one painting parameter, which helps to improve the efficiency and accuracy of editing and generating painting images. At the same time, providing different methods for obtaining user interactive dialogue data can enrich the user's dialogue input method, thereby improving the efficiency of human-computer interaction; by calculating the similarity score, the model parameters of the dialogue parameter extraction model are optimized and adjusted, and the parameter extraction accuracy of the dialogue parameter extraction model is further improved; different interactive dialogue data calculation methods are used for different rounds, which also improves the accuracy of similarity score calculation. Furthermore, according to the method of asking questions to interactively supplement parameters, the user's image editing needs can be more accurately understood and the normal operation of the target painting pipeline can be guaranteed, further improving the accuracy of target painting image generation.

[0225] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0226] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0229] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0230] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An interactive editing method for an image, characterized in that: The interactive editing method comprises: Obtain at least one round of interactive dialogue data input by the user; Inputting at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and outputting a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill; Based on the multiple target painting skills and the corresponding at least one painting parameter, at least one target painting pipeline is called, and a target painting image is edited and generated through the target painting pipeline.

2. The interactive editing method according to claim 1, characterized in that: The dialogue parameter extraction model is trained by the following steps: Acquire multiple sets of sample interactive dialogue data including dialogue data of different rounds and sample images corresponding to each round of dialogue data; Input each group of sample interactive dialogue data into the constructed multimodal large language model, and output sample painting skills and sample painting parameters; Generate a training target image based on the sample painting skills and the sample painting parameters; Detect whether there is an image where the training target image is inconsistent with the corresponding sample image; If so, adjust the model parameters of the multimodal large language model until each obtained training target image is consistent with the corresponding sample image, determine that the training of the multimodal large language model is completed, and obtain the dialogue parameter extraction model.

3. The interactive editing method according to claim 2, characterized in that: Multiple sets of sample interactive dialogue data are generated through the following steps: Obtain the painting models and painting pipelines used in the historical image editing process, and determine the input parameters and functional description information corresponding to each painting model and painting pipeline; Based on the input parameters and functional description information corresponding to each painting model and painting pipeline, multiple sets of sample interactive dialogue data are generated.

4. The interactive editing method according to claim 3, characterized in that: The method generates multiple sets of sample interactive dialogue data based on the input parameters and functional description information corresponding to each painting model and painting pipeline, including: For each input parameter and function description information obtained, multiple candidate painting images are generated; For each candidate painting image, generate a plurality of editing instructions for the candidate painting image; For each candidate painting image, multiple editing instructions corresponding to the candidate painting image are randomly combined according to a preset dialogue round ratio to obtain multiple groups of sample interactive dialogue data corresponding to the candidate painting image.

5. The interactive editing method according to claim 4, characterized in that: The multiple groups of sample interactive dialogue data include single-round dialogue data, multi-round dialogue data greater than a preset dialogue round, and multi-round dialogue data less than a preset dialogue round.

6. The interactive editing method according to claim 2, characterized in that: The interactive editing method further comprises: For each set of sample interactive dialogue data, calculating a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data; Determine the number of conversation data having a corresponding similarity score less than a preset similarity score threshold in the plurality of groups of sample interactive conversation data; If the number of dialogue data having a similarity score less than a preset similarity score threshold is greater than a preset number threshold, the model parameters of the dialogue parameter extraction model are updated and optimized.

7. The interactive editing method according to claim 6, characterized in that: The step of calculating, for each set of sample interactive dialogue data, a similarity score between a training target image generated by the sample interactive dialogue data and the sample interactive dialogue data includes: For each set of sample interactive dialogue data, if the current dialogue is an initial dialogue round in the sample interactive dialogue data, the similarity score is calculated based on the sample interactive dialogue data and the obtained training target image; For each set of sample interactive dialogue data, if the current dialogue is not an initial dialogue round in the sample interactive dialogue data, determine multiple dialogue rounds of interactive dialogue data based on the sample interactive dialogue data of the current dialogue round and the sample interactive dialogue data of the dialogue round before the current dialogue round; The similarity score is calculated based on the multiple rounds of interactive dialogue data and the training target image obtained in the current dialogue round.

8. The interactive editing method according to claim 1, characterized in that: The step of editing and generating a target painting image through the target painting pipeline includes: Detecting whether the current at least one drawing parameter includes all parameters required by the target drawing pipeline; If the current at least one drawing parameter includes all parameters required by the target drawing pipeline, the target drawing image is edited and generated through the target drawing pipeline based on the at least one drawing parameter.

9. The interactive editing method according to claim 8, characterized in that: The generating the target painting image by editing the target painting pipeline further includes: If the current at least one drawing parameter does not include all the parameters required by the target drawing pipeline, generating drawing parameter question information, and prompting the user to input the missing supplementary drawing parameters based on the drawing parameter question information; When it is determined that the at least one drawing parameter and the at least one supplementary drawing parameter input by the user include all parameters required by the target drawing pipeline, the target drawing image is edited and generated by the target drawing pipeline.

10. The interactive editing method according to claim 1, characterized in that: The step of obtaining at least one round of interactive dialogue data input by the user includes: If the interactive dialogue data input by the user is text information, generating at least one round of interactive dialogue data based on the text information; If the interactive dialogue data input by the user is voice information, at least one round of interactive dialogue data is generated after the voice information input by the user is converted into text information.

11. The interactive editing method according to claim 1, characterized in that: The target painting skills include at least one of the following: Drawing skills, image magnification skills, image style conversion skills, image partial redrawing skills, image optimization skills, image background update skills.

12. An interactive editing device for an image, characterized in that: The interactive editing device comprises: A data acquisition module, used to acquire at least one round of interactive dialogue data input by the user; A parameter output module, used to input at least one round of interactive dialogue data into a pre-trained dialogue parameter extraction model, and output a plurality of target painting skills and at least one painting parameter corresponding to each target painting skill; The image generation module is used to call at least one target painting pipeline based on the multiple target painting skills and the corresponding at least one painting parameter, and edit and generate a target painting image through the target painting pipeline.

13. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the interactive editing method of images as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interactive editing method of an image as claimed in any one of claims 1 to 11 are executed.

Images