Engineering application optimization method and device based on text graph, equipment and storage medium

Abstract

The embodiment of the invention provides an engineering application optimization method and device based on a text graph, equipment and a storage medium, and the method comprises the steps: inputting an original user instruction into a text graph model, and generating a first image; inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result; if the quality inspection result of the first image is that the quality inspection is passed, taking the first image as a target image; and if the quality inspection result of the first image is that the quality inspection is not passed, obtaining a second image as a target image according to an optimized user instruction obtained by optimizing the original user instruction, the text graph model and the image quality inspection model. By using the method, automatic quality inspection is performed on the generated image, and the quality of the generated image is monitored in real time. According to the quality inspection result, the user instruction is intelligently optimized, and the optimized instruction is used to tune the text graph model, so that the quality of the generated image is quickly improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an engineering application optimization method, device, equipment and storage medium based on a Wensheng diagram. Background Art

[0002] Text-to-Image technology is a specific application of Artificial Intelligence Generated Content (AIGC), which aims to generate corresponding images based on the text description provided by the user. The goal of this technology is to achieve efficient conversion between text and image, so that computers can understand and visualize information in human language. With the rapid development of AIGC technology, especially in the field of text-to-image, although certain breakthroughs have been made, there are still many defects and shortcomings, such as unstable generation quality, limitations in context understanding, and insufficient processing of abstract concepts.

[0003] At present, the optimization of the generation quality of text-based image applications is mainly achieved through the following methods: 1) Training data and strategy enhancement: improving the quality of training data and optimizing the model training process; 2) Model architecture optimization: enhancing the model generation capability by improving the model architecture; 3) Prompt word engineering: expanding modifiers for the prompt words entered by the user, adjusting the weights of the modifiers and injecting time steps to optimize the text prompts, thereby more accurately controlling image generation.

[0004] However, existing technologies mainly focus on optimizing the capabilities of the text graph model itself, which usually requires a long period of time and a large amount of computing resources, and is difficult to meet the needs of rapid application implementation. In addition, the third method of directly optimizing the prompt words has relatively limited effects in optimizing generation quality, especially for prompt words containing special semantics and abstract concepts. Summary of the invention

[0005] The present invention provides an engineering application optimization method, device, equipment and storage medium based on Vincent graphs, which realizes intelligent optimization of user instructions according to quality inspection results, and uses the optimized instructions to adjust the Vincent graph model, thereby quickly improving the quality of generated images and enhancing user experience.

[0006] In a first aspect, an embodiment of the present invention provides an engineering application optimization method based on a Wensheng diagram, comprising:

[0007] Inputting the original user instruction into the Wensheng graph model to generate a first image;

[0008] Inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result;

[0009] If the quality inspection result of the first image is that the quality inspection passes, taking the first image as the target image;

[0010] If the quality inspection result of the first image is that the quality inspection fails, a second image is obtained as a target image according to an optimized user instruction obtained by optimizing the original user instruction, the text graph model and the image quality inspection model.

[0011] In a second aspect, an embodiment of the present invention provides an engineering application optimization device based on a Wensheng diagram, comprising:

[0012] A first generating module, used for inputting the original user instruction into the Wensheng graph model to generate a first image;

[0013] An image quality inspection module, used for inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result;

[0014] A quality inspection passing module, configured to use the first image as a target image if the quality inspection result of the first image is that the first image passes the quality inspection;

[0015] The quality inspection failure module is used to obtain a second image as a target image according to the optimized user instruction obtained by optimizing the original user instruction, the text graph model and the image quality inspection model if the quality inspection result of the first image is that the quality inspection fails.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively coupled to the at least one processor;

[0019] Wherein, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the engineering application optimization method based on the Wensheng diagram as described in the embodiment of the first aspect.

[0020] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the engineering application optimization method based on the Vincent diagram as described in the embodiment of the first aspect.

[0021] The embodiment of the present invention provides an engineering application optimization method, device, equipment and storage medium based on a Wensheng graph, the method comprising: firstly, inputting the original user instruction into the Wensheng graph model to generate a first image; secondly, inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result; if the first image quality inspection result is that the quality inspection passes, the first image is used as the target image; if the first image quality inspection result is that the quality inspection fails, then according to the optimized user instruction obtained after optimizing the original user instruction, the Wensheng graph model and the image quality inspection model, a second image is obtained as the target image. The above technical scheme adds a quality inspection link for the image generated by the Wensheng graph model in the engineering link of generating images based on text, and monitors the quality of the generated image in real time by automatically inspecting the generated image. According to the quality inspection results, the user instruction is intelligently optimized, and the Wensheng graph model is adjusted using the optimized instruction, so as to quickly improve the quality of the generated image and improve the user experience.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a process flow of an engineering application optimization method based on a Wensheng diagram provided in the first embodiment of the present invention;

[0025] Figure 2 A schematic flow chart of another engineering application optimization method based on a Wensheng diagram provided in the second embodiment of the present invention;

[0026] Figure 3 A schematic diagram of executing an engineering application optimization method based on a Wensheng diagram in a certain application scenario provided in the second embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the structure of an engineering application optimization device based on a Wensheng diagram provided in the third embodiment of the present invention;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] With the rapid development of AIGC technology, especially in the field of text-based graphics, although certain breakthroughs have been made, there are still many defects and shortcomings, such as: 1) Unstable generation quality: Users usually need to try the same instruction multiple times to obtain higher quality images. Especially when dealing with complex scenes and images with rich details, the clarity and consistency of the generated results often cannot meet user expectations at one time; 2) Limitations of context understanding: When parsing complex text descriptions, existing models often cannot fully understand the implicit semantics and contextual relationships, which may cause the generated images to not match the text content, affecting the user experience; 3) Insufficient handling of abstract concepts: When generating abstract or creative images, the expressiveness of the model is limited, and it is difficult to accurately capture and express the user's intentions.

[0032] At present, the optimization of the generation quality of text-based image applications is mainly achieved through the following methods: 1) Training data and strategy enhancement: improving the quality of training data and optimizing the model training process; 2) Model architecture optimization: enhancing the model generation capability by improving the model architecture; 3) Prompt word engineering: expanding modifiers for the prompt words entered by the user, adjusting the weights of the modifiers and injecting time steps to optimize the text prompts, thereby more accurately controlling image generation.

[0033] However, existing technologies mainly focus on optimizing the capabilities of the text graph model itself, which usually requires a long period of time and a large amount of computing resources, and is difficult to meet the needs of rapid application implementation. In addition, the third method of directly optimizing the prompt words has relatively limited effects in optimizing generation quality, especially for prompt words containing special semantics and abstract concepts.

[0034] Embodiment 1

[0035] Figure 1 The present invention provides a flowchart of a method for optimizing engineering applications based on a text map, which is applicable to the case of improving the quality of image generation based on text. The method can be performed by an engineering application optimization device based on a text map, which can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the engineering application optimization method based on the Wensheng diagram provided in the first embodiment may specifically include the following steps:

[0036] S101: Input an original user instruction into a text graph model to generate a first image.

[0037] In this embodiment, the text graph model can be specifically understood as a multimodal deep learning model that can generate an image that matches the description from the text description. The original user instruction can be specifically understood as a user instruction for describing what kind of image to be generated. Specifically, the original user instruction is input into the text graph model, and the text graph model generates a corresponding image based on the original user instruction, which is recorded as the first image.

[0038] S102: Input the first image into a predetermined image quality inspection model to obtain a first image quality inspection result.

[0039] Considering that the quality or content of the first image generated by the Vincent graph model is unstable and does not meet the user's expectations, it is different from the prior art that the quality of the image is improved by optimizing the Vincent graph model; in this embodiment, the focus is on optimizing the quality of the Vincent graph in the engineering application link, and an automated image quality inspection mechanism is introduced after the image is generated based on the Vincent graph model to improve the quality of the image. The image quality inspection model can be specifically understood as a model for detecting the quality of the image. The image quality inspection result is obtained through the image quality inspection model to determine whether the generated image quality is better or worse. For the bad part, the prompt words will be optimized and then given to the Vincent graph model to regenerate the image.

[0040] In this embodiment, the image quality inspection model is predetermined, and the determination process of the image quality inspection model can be expressed as follows: obtain several multimodal models that can perform quality inspection on images, these multimodal models can understand and judge images, and these models can be existing models that can perform image quality inspection. In order to screen out models with higher image quality inspection accuracy from these models, some image test sets will be constructed. These image test sets are labeled, and the image test sets will be used to test which model has the best quality inspection effect from these multimodal models. For different attributes of the image, corresponding quality inspection scenes are set, and corresponding image test sets are constructed. For at least one quality inspection scene, an image test set corresponding to the quality inspection scene is constructed; based on the image test set, each quality inspection model to be evaluated is evaluated, and the quality inspection model to be evaluated corresponding to the highest accuracy in each evaluation result is used as the image quality inspection model corresponding to the quality inspection scene.

[0041] In this step, the first image is input into the determined image quality inspection model, and the image quality inspection model outputs the result of the quality inspection of the first image, which is recorded as the first image quality inspection result. The first image quality inspection result includes whether the first image passes the quality inspection or fails the quality inspection. If the quality inspection fails, the quality inspection result may also include the reason for the failure.

[0042] S103: If the quality inspection result of the first image is that the quality inspection passes, the first image is used as the target image.

[0043] The target image can be specifically understood as the image finally generated. In this embodiment, if the quality inspection result of the first image is that the quality inspection passes, the first image is used as the target image finally generated.

[0044] S104: If the quality inspection result of the first image is that the quality inspection fails, a second image is obtained as a target image according to the optimized user instruction, the text graph model and the image quality inspection model obtained after optimizing the original user instruction.

[0045] In this embodiment, for the first image that fails the quality inspection, it is necessary to perform problem attribution analysis to identify the specific error type, and further utilize the capabilities of the large language model to intelligently optimize the original user instructions. The original user instructions and the attribution for the failure of the quality inspection are input into the large language model, and the large language model is informed based on the instruction optimization prompt that the original user instructions need to be optimized based on the attribution for the failure of the quality inspection. The original user instructions are parsed based on the large language model to extract key features and semantic information to understand the user's intent and obtain user intent information. Based on the user intent information and the attribution for the failure of the quality inspection, an optimization strategy is used to optimize the original user instructions to generate optimized optimized user instructions. These optimization strategies include adding modifiers, adjusting the details of the description, and re-structuring the sentences to better guide the model to generate images that meet expectations.

[0046] Following the above description, the optimized user instruction is sent to the Wensheng graph model for secondary generation, and the generated image is recorded as the second image. The second image is input into the image quality inspection model for secondary quality inspection, and the quality inspection result is obtained, which is recorded as the second image quality inspection result. If the quality inspection result of the second image is that the quality inspection passes, the second image can be fed back to the user as the target image. If the quality inspection result of the second image is that the quality inspection fails, that is, both quality inspections fail, a prompt message is generated, which is recorded as the quality inspection failure prompt message, and the quality inspection failure prompt message includes the quality inspection failure attribution. The cause of the error is fed back to the user, and the user can modify and optimize the user instruction by himself to generate a better quality image.

[0047] It should be noted that the image quality inspection link added to the engineering link provided in this embodiment can not only be used in Wensheng image applications, but can also be quickly expanded to other AIGC applications such as Wensheng video.

[0048] The embodiment of the present invention provides an engineering application optimization method based on a text graph, which adds a quality inspection link for images generated by a text graph model in the engineering link of generating images based on text, and monitors the quality of the generated images in real time by automatically inspecting the generated images. According to the quality inspection results, the user instructions are intelligently optimized, and the text graph model is adjusted using the optimized instructions, so as to quickly improve the quality of the generated images and enhance the user experience.

[0049] As an optional embodiment of the embodiment of the present invention, based on the above embodiment, the steps of determining the image quality inspection model may be optimized, including:

[0050] a1) Obtain each quality inspection model to be evaluated.

[0051] The quality inspection model to be evaluated is a model for performing quality inspection on images.

[0052] In this embodiment, several multimodal models that can perform image quality inspection are obtained. These multimodal models can understand and judge images, and the models used for image quality inspection are recorded as quality inspection models to be evaluated. This step is used to obtain existing models that can perform image quality inspection as quality inspection models to be evaluated.

[0053] b1) For at least one quality inspection scenario, construct an image test set corresponding to the quality inspection scenario.

[0054] In this embodiment, the quality inspection scenario can be specifically understood as the quality inspection of the image attributes in which aspects. For example, the quality inspection scenario can be scenarios such as entity consistency, style consistency, and image clarity. Among them, entity consistency means that the entity in the instruction input by the user is consistent with the entity in the actual generated image. Style consistency means that for the picture style, the image style that the user wants to generate is consistent with the style of the actual generated image. For example, if the user wants to generate a traditional Chinese style, anime, or a realistic image, then the actual generated image is traditional Chinese style, anime, or realistic, and whether it can correspond to the user's needs. Image clarity refers to the sharpness of the edge changes of the generated image details, such as clear distinction between image lines, clear line edge contours, and clear light and dark levels of the image.

[0055] In this embodiment, in order to select the model with the highest accuracy from the various quality inspection models to be evaluated as the image quality inspection model, some image test sets are constructed. These image test sets are labeled, and the quality inspection models to be evaluated are tested based on these image test sets, and then the model with the best quality inspection effect is selected from the many quality inspection models to be evaluated. Since different quality inspection scenarios have different focuses on images, corresponding quality inspection scenarios are set for different attributes of the image, and corresponding image test sets are constructed. Different image test sets corresponding to different quality inspection scenarios are constructed and labeled differently.

[0056] Exemplarily, for entity consistency, a corresponding image test set is constructed, which includes pictures and label files. The label files correspond to the pictures one by one, and the label content is {"prompt":"A possible description of the picture","result":1}, where prompt is a description of the picture, and result is a score for judging whether the picture and the prompt description are consistent. 1 indicates that the entity is consistent, and 0 indicates that the entity is inconsistent.

[0057] c1) Evaluate each quality inspection model to be evaluated based on the image test set, and use the quality inspection model to be evaluated with the highest accuracy in each evaluation result as the image quality inspection model corresponding to the quality inspection scenario.

[0058] In this embodiment, the image quality inspection model can be specifically understood as a model for performing quality inspection on images generated by the Wensheng graph model. For different quality inspection scenarios, the image test set constructed for the quality inspection scenario is used to evaluate each quality inspection model to be evaluated, and the model with the highest accuracy is selected as the image quality inspection model corresponding to the quality inspection scenario. Accuracy calculation method: Accuracy = number of correctly detected images / total number of images. It can be understood that for different quality inspection scenarios, the corresponding image quality inspection models can be screened out respectively for subsequent quality inspection of images generated by the Wensheng graph model.

[0059] The above technical solution specifies the steps for determining the image quality inspection model. By constructing an image test set, each quality inspection model to be evaluated is screened, and the model with the highest accuracy is selected as the image quality inspection model, which is used for quality inspection of images generated by subsequent cultural image models, providing a basis for adding quality inspection links in the cultural image engineering application chain.

[0060] As another optional embodiment of the embodiment of the present invention, on the basis of the above embodiment, it can be optimized that after obtaining the target image, the method further includes: storing the image quality inspection result to optimize the viz-graph model based on the image quality inspection result.

[0061] In this embodiment, a storage mechanism is established to store the image quality inspection results of the quality inspection images for subsequent model optimization analysis. Regardless of whether the quality inspection results of the images generated by the text graph model pass or fail, and the reasons for failure, they will all be stored for optimization of the text graph model.

[0062] The above technical solution adds a storage mechanism for image quality inspection results, providing basic data for subsequent optimization of the text graph model.

[0063] Embodiment 2

[0064] Figure 2 A flow chart of another method for optimizing engineering applications based on a Vincent graph provided in Embodiment 2 of the present invention. This embodiment is a further optimization of the above-mentioned embodiment. In this embodiment, “inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result” is further optimized, and “obtaining a second image as a target image according to the optimized user instruction obtained after optimizing the original user instruction, the Vincent graph model and the image quality inspection model” is optimized.

[0065] like Figure 2 As shown, this embodiment 2 provides an engineering application optimization method based on a Wensheng diagram, which specifically includes the following steps:

[0066] S201: Input an original user instruction into a text graph model to generate a first image.

[0067] S202: Determine an image quality assessment instruction for assessing image quality according to quality inspection requirements.

[0068] Among them, the image quality assessment instructions at least include: image clarity assessment instructions, entity consistency assessment instructions or style consistency assessment instructions. The image clarity assessment instructions can be specifically understood as instructions for evaluating image clarity, such as clear distinction between image lines, clear line edge contours, and clear light and dark levels of the image. The entity consistency assessment instructions can be specifically understood as instructions for evaluating whether the entities in the generated image are consistent with the entities described in the user instructions. The style consistency assessment instructions can be specifically understood as instructions for evaluating whether the style of the image that the user wants to generate is consistent with the style of the actual generated image, such as if the user wants to generate a Chinese style, anime, or a realistic image, then determine whether the actually generated image is Chinese style, anime, or realistic, and evaluate whether it corresponds to the user's needs.

[0069] In this embodiment, according to what kind of quality inspection scenarios need to be integrated in the project, that is, according to the quality inspection requirements, the image quality assessment instructions for evaluating the image quality are determined. For example, the quality inspection of entity consistency can be performed, and the quality inspection of image style consistency can also be performed. It can also be asked to perform entity consistency detection first, and then let it make a judgment on style consistency or let it make a judgment on character details. It can be understood that the quality inspection model and image quality assessment instructions selected in the entire text-based image link will correspond to the actual quality inspection scenario.

[0070] Taking entity consistency inspection as an example, the evaluation instruction can be described as: You are a proferssional text-to-image alignment evaluator. I will provide a input text prompt and a corresponding image generated by a text-to-image model. It should be noted that it is required to only evaluate whether the entity in the text exists in the image. If it exists, it returns {"result":1}, if it does not exist, it returns, {"result":0,"error_reason":""}. Please answer as asked and do not output redundant text. \'input text prompt':{prompt}. Among them, prompt is the description of the image, result is the score for judging whether the image and prompt description are consistent, 1 means the entity is consistent, 0 means the entity is inconsistent, and error_reason indicates the error reason for entity inconsistency.

[0071] S203: Input the image quality assessment instruction and the first image into the image quality inspection model to obtain a first image quality inspection result.

[0072] In this embodiment, according to the actual engineering needs, image quality assessment instructions are set, such as clarity, entity consistency, style consistency, etc., to automatically inspect the generated images, identify potential problems in a timely manner, and directly output the images that pass the quality inspection to the user. Specifically, the image quality assessment instruction and the first image are input into the image quality inspection model, wherein the first image represents the image generated by the Wensheng graph model, and the image quality assessment instruction indicates which aspect of the quality assessment is to be performed on the first image. Based on the image quality assessment instruction, it is determined which image quality inspection model is used to inspect the first image, and the quality inspection result of the first image is obtained.

[0073] S204: If the quality inspection result of the first image is that the quality inspection passes, the first image is used as the target image.

[0074] Specifically, if the quality inspection result of the first image is that the quality inspection passes, the first image is directly output to the user as the target image.

[0075] S205: If the quality inspection result of the first image is that the quality inspection fails, the original user instruction is optimized based on the large language model to obtain an optimized user instruction.

[0076] In this embodiment, for the first image that fails the quality inspection, it is necessary to perform problem attribution analysis, identify the specific error type, and further utilize the capabilities of the large language model to intelligently optimize the original user instructions. For example, for clarity and entity consistency errors, secondary generation and user instruction optimization processing are required. For authenticity errors, the original user instructions need to be analyzed for intent and style. If the user expects a non-realistic style such as science fiction, animation, etc., then the authenticity error is not processed, otherwise secondary generation and user instruction optimization processing are required.

[0077] Optionally, if the image quality inspection result is that the quality inspection fails, the first image quality inspection result also includes a reason for the quality inspection failure.

[0078] As a specific implementation method, the steps of optimizing the original user instruction based on the large language model to obtain the optimized user instruction include:

[0079] a2) Input instruction optimization tips, original user instructions, and reasons for quality inspection failure into the large language model.

[0080] Among them, the instruction optimization prompt is used to guide the optimization of the original user instruction based on the attribution of quality inspection failure. This step is used to input the instruction optimization prompt, the original user instruction and the attribution of quality inspection failure into the large language model, and inform the large language model based on the instruction optimization prompt that the original user instruction needs to be optimized based on the attribution of quality inspection failure.

[0081] b2) Parse the original user instructions based on the large language model to obtain user intent information.

[0082] Specifically, the original user instructions are parsed based on the large language model to extract key features and semantic information to understand the user's intention and obtain user intention information.

[0083] c2) Generate optimized user instructions based on user intent information and quality inspection failure attribution.

[0084] In this embodiment, based on the user intent information and the reasons for failing the quality inspection, optimization strategies are used to optimize the original user instructions to generate optimized user instructions. These optimization strategies include adding modifiers, adjusting the details of the description, and re-structuring the sentences to better guide the model to generate images that meet expectations.

[0085] For example, if the reason why the first image quality inspection fails is an entity consistency error, the image quality inspection result will not only feedback that the image quality inspection fails, but also return the reason for the error. For example, the reason for the error may be that the image does not contain the xx entity. By returning the specific reason, the user's instructions can be optimized according to the cause of the error. If the gender or character in the entity is wrong, some of the user's instructions can be emphasized or feature extracted. The user's instructions may include a long paragraph. Only some key and clear information needs to be extracted and given to the text image model, and the generated image effect will be better.

[0086] The above technical solution specifies the steps of optimizing the original user instructions to obtain optimized user instructions. Compared with simple prompt word engineering optimization, the above technical solution is more targeted and comprehensive. It only performs post-processing operations on images that fail quality inspection. It combines the original user instructions and quality inspection results, and uses a large language model to intelligently optimize user instructions. This can effectively solve the problem of poor quality of images generated by user instructions containing special semantics and abstract concepts.

[0087] S206: Input the optimization user instruction into the text graph model to generate a second image.

[0088] In this embodiment, the optimized user instruction is sent to the text graph model for secondary generation, and the generated image is recorded as a second image.

[0089] S207: Input the second image into the image quality inspection model to obtain a second image quality inspection result.

[0090] Specifically, the second image is input into the image quality inspection model, and a second quality inspection is performed to obtain a quality inspection result of this time, which is recorded as the second image quality inspection result.

[0091] S208: If the quality inspection result of the second image is passed, the second image is used as the target image.

[0092] Specifically, if the quality inspection result of the second image is that the quality inspection passes, the second image can be fed back to the user as the target image.

[0093] S209: If the quality inspection result of the second image is that the quality inspection fails, the second image is used as the target image and instruction optimization prompt information is generated and fed back to the user.

[0094] The instruction optimization prompt information includes that the second image effect is not good, and it is recommended to enter a clearer instruction. In this embodiment, if the second image quality inspection result is that the quality inspection fails, that is, both quality inspections fail, a prompt information is generated, recorded as the instruction optimization prompt information, prompting the user that the instruction currently entered by the user may result in a poor generated image effect, and it is recommended to enter a clearer instruction. The instruction optimization prompt information is fed back to the user, and the user can modify and optimize the user instruction by himself to generate an image with better quality.

[0095] The above technical solution specifies the steps of obtaining the quality inspection result of the first image and the steps of obtaining the second image as the target image. Based on the AIGC application optimization of the large model, through automated image quality inspection and intelligent optimization of user instructions, efficient management and quality control of the Vincent image generation process are achieved. Since the quality inspection optimization process is implemented in the underlying link, the quality of the generated image can be significantly improved without the user's perception. It can effectively improve the stability of the quality of Vincent image generation, thereby improving the user experience and promoting the application process of Vincent image technology. Compared with training data and strategy enhancement and model architecture optimization, this technical solution can quickly improve the user experience without consuming a long time and a large amount of computing resources. At the same time, it has wide applicability and reference value in a variety of AIGC applications.

[0096] In order to more clearly describe the engineering application optimization method based on text map provided by the embodiment of the present invention, the text-based image generation in a certain application scenario is described as an example. Figure 3 A schematic diagram of an execution of an engineering application optimization method based on a Wensheng diagram in a certain application scenario provided in the second embodiment of the present invention, such as Figure 3 As shown, the quality inspection models to be evaluated are arranged in the form of a list of quality inspection models to be evaluated, including model A, model B and model C respectively. The models in the list of quality inspection models to be evaluated are tested based on the constructed image test set, and the model with the highest accuracy is selected as the image quality inspection model. The engineering application link of generating images based on text can be described as inputting the original user instructions into the text image model to generate an image, inputting the image and the image quality assessment instructions into the image quality inspection model, and obtaining the image quality inspection result. If the image quality inspection result is that the quality inspection passes, the image is finally generated. If the image quality inspection result is that the quality inspection fails, the attribution of the quality inspection failure is determined, and the attribution of the quality inspection failure and the original user instructions are input into the large language model to obtain the optimized user instructions. The optimized user instructions are input into the image generation model to obtain the secondary generated image and then perform secondary quality inspection.

[0097] Embodiment 3

[0098] Figure 4Schematic diagram of the structure of an engineering application optimization device based on a text map provided in Embodiment 3 of the present invention. The device can be applied to the case where the quality of image generation is improved based on text generation. The engineering application optimization device based on a text map can be configured in an electronic device, such as Figure 4 As shown, the device includes: a first generation module 31, an image quality inspection module 32, a quality inspection pass module 33 and a quality inspection fail module 34; wherein,

[0099] A first generating module 31 is used to input the original user instruction into the Wensheng graph model to generate a first image;

[0100] An image quality inspection module 32, used to input the first image into a predetermined image quality inspection model to obtain a first image quality inspection result;

[0101] The quality inspection passing module 33 is used to use the first image as the target image if the quality inspection result of the first image is that the quality inspection passes;

[0102] The quality inspection failure module 34 is used to obtain a second image as a target image according to the optimized user instructions, the text graph model and the image quality inspection model obtained by optimizing the original user instructions if the quality inspection result of the first image is that the quality inspection fails.

[0103] The above technical solution adds a quality inspection link for images generated by the text-based image generation model to the engineering application link of text-based image generation. By automatically inspecting the generated images, the quality of the generated images can be monitored in real time. According to the quality inspection results, user instructions are intelligently optimized, and the text-based image model is adjusted using the optimized instructions, thereby quickly improving the quality of generated images and enhancing user experience.

[0104] Optionally, the device further comprises a quality inspection model determination module, the quality inspection model determination module being specifically used for

[0105] Obtain each quality inspection model to be evaluated, where the quality inspection model to be evaluated is a model for performing quality inspection on an image;

[0106] For at least one quality inspection scenario, build an image test set corresponding to the quality inspection scenario;

[0107] Each quality inspection model to be evaluated is evaluated based on the image test set, and the quality inspection model to be evaluated corresponding to the highest accuracy in each evaluation result is used as the image quality inspection model corresponding to the quality inspection scenario.

[0108] Optionally, the image quality inspection module 32 is specifically used for:

[0109] Determine image quality assessment instructions for assessing image quality based on quality inspection requirements;

[0110] The image quality assessment instruction and the first image are input into the image quality inspection model to obtain the first image quality inspection result.

[0111] Optionally, the quality inspection failure module 34 is specifically used for:

[0112] An instruction optimization unit, used for optimizing the original user instruction based on the large language model to obtain an optimized user instruction;

[0113] An image generating unit, used for inputting the optimization user instruction into the Wensheng graph model to generate a second image;

[0114] An image quality inspection unit, used for inputting the second image into the image quality inspection model to obtain a second image quality inspection result;

[0115] A quality inspection passing unit, configured to use the second image as a target image if the quality inspection result of the second image is that the second image passes the quality inspection;

[0116] The quality inspection failure unit is used to use the second image as the target image and generate instruction optimization prompt information to feedback to the user if the quality inspection result of the second image is that the second image fails the quality inspection. The instruction optimization prompt information includes that the second image effect is not good and it is recommended to enter a clearer instruction.

[0117] Optionally, if the image quality inspection result is that the quality inspection fails, the first image quality inspection result also includes a reason for failing the quality inspection;

[0118] Units that fail quality inspection are specifically used for:

[0119] Input the instruction optimization tips, original user instructions, and quality inspection failure attribution into the large language model. The instruction optimization tips are used to guide the optimization of the original user instructions based on the quality inspection failure attribution.

[0120] Parse the original user instructions based on the large language model to obtain user intent information;

[0121] Generate optimized user instructions based on user intent information and quality inspection failure attribution.

[0122] Optionally, the image quality assessment instruction includes at least: an image clarity assessment instruction, an entity consistency assessment instruction or a style consistency assessment instruction.

[0123] Optionally, the device further comprises a model optimization module, which is used, after obtaining the target image, to:

[0124] The image quality inspection results are stored to optimize the text graph model based on the image quality inspection results.

[0125] The engineering application optimization device based on the Vincent diagram provided in the embodiment of the present invention can execute the engineering application optimization method based on the Vincent diagram provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0126] Embodiment 4

[0127] Figure 5 A schematic diagram of the structure of an electronic device provided for Embodiment 4 of the present invention. The electronic device is intended to represent various forms of digital computers, such as desktop computers, servers, smart phones, etc. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0128] like Figure 5 As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0129] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as the engineering application optimization method based on the Wensheng diagram.

[0131] In some embodiments, the engineering application optimization method based on the Wensheng diagram can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the engineering application optimization method based on the Wensheng diagram described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the engineering application optimization method based on the Wensheng diagram in any other appropriate manner (for example, by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0134] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0138] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the engineering application optimization method based on the Wensheng diagram as provided in any embodiment of the present invention.

[0139] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code for the operation of the present disclosure, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0141] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An engineering application optimization method based on Wensheng diagram, characterized in that: include: Inputting the original user instruction into the Wensheng graph model to generate a first image; Inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result; If the quality inspection result of the first image is that the quality inspection passes, taking the first image as the target image; If the quality inspection result of the first image is that the quality inspection fails, a second image is obtained as a target image according to an optimized user instruction obtained by optimizing the original user instruction, the text graph model and the image quality inspection model.

2. The method according to claim 1, characterized in that The step of determining the image quality inspection model includes: Obtain each quality inspection model to be evaluated, where the quality inspection model to be evaluated is a model for performing quality inspection on an image; For at least one quality inspection scenario, construct an image test set corresponding to the quality inspection scenario; Each of the quality inspection models to be evaluated is evaluated based on the image test set, and the quality inspection model to be evaluated corresponding to the highest accuracy in each evaluation result is used as the image quality inspection model corresponding to the quality inspection scenario.

3. The method according to claim 1, characterized in that The step of inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result includes: Determine image quality assessment instructions for assessing image quality based on quality inspection requirements; The image quality assessment instruction and the first image are input into the image quality inspection model to obtain the first image quality inspection result.

4. The method according to claim 1, characterized in that: The step of obtaining a second image as a target image according to the optimized user instruction obtained by optimizing the original user instruction, the cultural graph model, and the image quality inspection model comprises: Optimizing the original user instruction based on the large language model to obtain an optimized user instruction; Inputting the optimization user instruction into the Wensheng graph model to generate a second image; Inputting the second image into the image quality inspection model to obtain a second image quality inspection result; If the quality inspection result of the second image is that the quality inspection passes, taking the second image as the target image; If the quality inspection result of the second image is that it fails the quality inspection, the second image is used as the target image and instruction optimization prompt information is generated and fed back to the user. The instruction optimization prompt information includes that the second image effect is not good and it is recommended to enter a clearer instruction.

5. The method according to claim 4, characterized in that If the image quality inspection result is that the quality inspection fails, the first image quality inspection result also includes the reason for the quality inspection failure; The step of optimizing the original user instruction based on the large language model to obtain an optimized user instruction includes: Inputting the instruction optimization prompt, the original user instruction and the attribution of the quality inspection failure into the large language model, wherein the instruction optimization prompt is used to guide the optimization of the original user instruction based on the attribution of the quality inspection failure; Parsing the original user instruction based on the large language model to obtain user intention information; The optimization user instruction is generated according to the user intention information and the attribution of the quality inspection failure.

6. The method according to claim 3, characterized in that The image quality assessment instruction at least includes: an image clarity assessment instruction, an entity consistency assessment instruction or a style consistency assessment instruction.

7. The method according to claim 1, characterized in that After obtaining the target image, the method further includes: The image quality inspection result is stored to optimize the text graph model based on the image quality inspection result.

8. An engineering application optimization device based on Wensheng diagram, characterized in that: include: A first generating module, used for inputting the original user instruction into the Wensheng graph model to generate a first image; An image quality inspection module, used for inputting the first image into a predetermined image quality inspection model to obtain a first image quality inspection result; A quality inspection passing module, configured to use the first image as a target image if the quality inspection result of the first image is that the first image passes the quality inspection; The quality inspection failure module is used to obtain a second image as a target image according to the optimized user instruction obtained by optimizing the original user instruction, the text graph model and the image quality inspection model if the quality inspection result of the first image is that the quality inspection fails.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the engineering application optimization method based on the Wensheng diagram as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to execute the engineering application optimization method based on the Vincent diagram according to any one of claims 1 to 7.

Images