Geometric image synthesis method, related device and system

Abstract

The invention discloses a geometric image synthesis method, a related device and a system, and relates to the technical field of image synthesis, and the geometric image synthesis method comprises the steps: obtaining a geometric image demand text described by a user in a natural language; generating an image synthesis code according to the geometric image demand text by using the code large language model; synthesizing a plurality of geometric images based on the image synthesis codes; verifying the plurality of geometric images in combination with the geometric image demand text, and determining the conformity of the current image composite code according to the verification result; and if the conformity is greater than or equal to a preset conformity threshold value, outputting a geometric image which is verified to be qualified, and / or outputting an image synthesis code. According to the geometric image synthesis method disclosed by the invention, full-process automation from geometric image demand understanding to geometric image synthesis is realized, a user only needs to describe the geometric image demand through a natural language and does not need to have professional programming ability and graphic knowledge, and the technical threshold is greatly reduced.

Description

Technical Field

[0001] This application relates to the field of image synthesis technology, and in particular to a geometric image synthesis method, related apparatus and related system. Background Technology

[0002] Geometric image synthesis technology, as an interdisciplinary application of computer vision and computer graphics, plays a crucial role in fields such as architectural design, industrial manufacturing, digital art, and scientific visualization.

[0003] Currently, the mainstream geometric image synthesis schemes are mainly manual ones. This means that users first need to be proficient in professional programming tools and graphics libraries (such as Processing, p5.js, OpenCV, Canvas API, etc.), then manually write detailed geometric image synthesis code according to the design intent, and finally synthesize the geometric image by executing the geometric image synthesis code. Whether the synthesized geometric image meets the initial requirements still depends on manual observation and verification.

[0004] Current geometric image synthesis solutions have a high technical threshold, requiring users to not only have solid programming skills but also a deep understanding of computer graphics principles. This constitutes a huge technical barrier for users and greatly limits the popularization and application of geometric image synthesis technology. Summary of the Invention

[0005] In view of this, this application provides a geometric image synthesis method, related apparatus, and related system to solve the problem that existing geometric image synthesis schemes are highly dependent on manual labor and have high technical barriers. The technical solution is as follows:

[0006] The first aspect of this application provides a geometric image synthesis method, comprising:

[0007] Obtain the user's text describing their geometric image requirements in natural language;

[0008] Using a code-large language model, code for synthesizing geometric images is generated based on the geometric image requirement text, thus obtaining the current image synthesis code;

[0009] Synthesize multiple geometric images based on the current image synthesis code;

[0010] The multiple geometric images are verified in conjunction with the geometric image requirement text, and the degree of conformity of the current image synthesis code with the geometric image requirement text is determined based on the verification results.

[0011] If the compliance degree is greater than or equal to the preset compliance degree threshold, then the verified geometric image is output, and / or the current image synthesis code is output, so that the geometric image can be synthesized directly based on the output code.

[0012] In one possible implementation, the geometric image synthesis method further includes:

[0013] If the compliance degree is less than the preset compliance degree threshold, then based on the problems existing in the unqualified geometric images among the multiple geometric images, cognitive-level and enlightening feedback prompts are generated, wherein the problems are discovered when verifying the multiple geometric images;

[0014] Using the code big language model, the current image synthesis code is modified according to the feedback prompts; the modified code is used as the current image synthesis code, and the steps of synthesizing multiple geometric images based on the current image synthesis code and subsequent steps are executed.

[0015] In one possible implementation, generating code for synthesizing the geometric image based on the geometric image requirement text using a code-large language model includes:

[0016] Convert the geometric image requirement text into a set of geometric constraints;

[0017] Using a code-large language model, code for synthesizing geometric images is generated based on the set of geometric constraints.

[0018] In one possible implementation, converting the geometric image requirement text into a set of geometric constraints includes:

[0019] Based on a pre-built geometric terminology mapping library, the geometric image requirement text is converted into a set of geometric constraints, wherein the geometric terminology mapping library includes multiple geometric shape types and attributes of each geometric shape type.

[0020] In one possible implementation, the conversion of the geometric image requirement text into a set of geometric constraints based on a pre-built geometric terminology mapping library includes:

[0021] Extract the target geometric shape type from the geometric image requirement text;

[0022] Obtain the attributes of the target geometry type from the geometry terminology mapping library;

[0023] Extract the attribute values ​​corresponding to the attributes of the target geometric shape type from the geometric image requirement text;

[0024] A set of geometric constraints is constructed based on the target geometric shape type, the attributes of the target geometric shape type, and the attribute values ​​corresponding to the attributes of the target geometric shape type.

[0025] In one possible implementation, the verification of the multiple geometric images in conjunction with the geometric image requirement text includes:

[0026] Using a visual large language model, combined with the geometric image requirement text, semantic verification is performed on the multiple geometric images respectively;

[0027] Based on the geometric image requirement text, a geometric verification is performed on each geometric image that passes the semantic verification.

[0028] In one possible implementation, the use of a visual large language model, combined with the geometric image requirement text, to perform semantic-level verification on the multiple geometric images includes:

[0029] For each of the multiple geometric images, a visual big language model is used, combined with the geometric image requirement text, to perform visual entity semantic verification, spatial relationship verification, and quantity conformity verification on the geometric image.

[0030] Specifically, the visual entity semantic verification is used to verify whether the types and attributes of independent visual entities in the geometric image meet the requirements described in the geometric image requirement text; the spatial relationship verification is used to verify whether the spatial relationships between different visual entities in the geometric image meet the requirements described in the geometric image requirement text; and the quantity compliance verification is used to verify whether the quantity of each type of visual entity in the geometric image meets the requirements described in the geometric image requirement text.

[0031] In one possible implementation, the step of combining the geometric image requirement text to perform geometric-level verification on each geometric image that passes semantic-level verification includes:

[0032] For each geometric image that passes semantic-level verification:

[0033] Based on the geometric image requirement text, determine the core geometric parameters and the target parameter values ​​of the core geometric parameters;

[0034] The actual parameter values ​​of the core geometric parameters are obtained by measuring the visual entities in the geometric image;

[0035] Based on the actual parameter values ​​and target parameter values ​​of the core geometric parameters, the geometric image is subjected to a precise conformity verification of the core geometric parameters.

[0036] In one possible implementation, determining the conformity of the current image synthesis code to the geometric image requirement text based on the verification result includes:

[0037] Geometric images that pass both semantic and geometric level verification are identified as geometric images that meet the requirements described in the geometric image requirement text.

[0038] The percentage of geometric images that meet the requirements described in the geometric image requirement text among the multiple geometric images is used as the degree of compliance of the current image synthesis code with the geometric image requirement text.

[0039] In one possible implementation, the geometric image synthesis method further includes:

[0040] If an error occurs during the generation of a geometric image based on the current image synthesis code, the current image synthesis code is modified using the code language model based on the error information generated during the generation of the geometric image.

[0041] A second aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program so that the electronic device can implement any of the above-described geometric image synthesis methods.

[0044] A third aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement any of the above-described geometric image synthesis methods.

[0045] The fourth aspect of this application provides a geometric image generation system, including: a requirement definition unit, a code generation unit, an image synthesis unit, an image verification unit, and a decision output unit;

[0046] The requirement definition unit is used to obtain the geometric image requirement text described by the user in natural language;

[0047] The code generation unit is used to generate code for synthesizing geometric images based on the geometric image requirement text using a code large language model, thereby obtaining the current image synthesis code;

[0048] The image synthesis unit is used to synthesize multiple geometric images based on the current image synthesis code;

[0049] The image verification unit is used to verify the multiple geometric images in conjunction with the geometric image requirement text, and to determine the degree of conformity of the current image synthesis code with the geometric image requirement text based on the verification results.

[0050] The decision output unit is used to output a qualified geometric image when the compliance degree is greater than or equal to a preset compliance degree threshold, and / or output the current image synthesis code so as to directly synthesize a geometric image based on the output code.

[0051] By employing the above technical solution, the geometric image synthesis method provided in this application first obtains the user's geometric image requirement text described in natural language. Then, using a code large language model, it generates code for synthesizing geometric images based on the geometric image requirement text, obtaining the current image synthesis code. Next, it synthesizes multiple geometric images based on the current image synthesis code. Then, it verifies the multiple geometric images in conjunction with the geometric image requirement text, and determines the conformity of the current image synthesis code to the geometric image requirement text based on the verification results. If the conformity is greater than or equal to a preset conformity threshold, it outputs the verified qualified geometric image and / or outputs the current image synthesis code (which can be directly synthesized based on the output code). The geometric image synthesis method provided in this application achieves full automation from understanding geometric image requirements to geometric image synthesis. The geometric image synthesis process does not require manual intervention. For users, since they only need to describe the geometric image requirements in natural language, they do not need professional programming skills, graphics knowledge, or complex drawing software operation skills, greatly reducing the technical threshold. At the same time, the geometric image synthesis method provided in this application significantly improves the efficiency of geometric image synthesis and reduces time and labor costs. In addition, this application first uses a code-large language model to generate code for synthesizing geometric images, and then synthesizes geometric images based on the code. This image synthesis strategy has significant advantages in terms of accuracy and scalability. This application also introduces a verification operation for the synthesized image and quantifies the verification result into a compliance degree. Decisions are made based on the compliance degree to output high-quality results. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0053] Figure 1 A schematic flowchart illustrating the geometric image synthesis method provided in this application embodiment;

[0054] Figure 2 This is a schematic diagram illustrating the process of verifying multiple geometric images by combining geometric image requirement text, as provided in an embodiment of this application.

[0055] Figure 3This is a schematic diagram of the structure of the geometric image synthesis system provided in the embodiments of this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0057] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] Current geometric image synthesis schemes are mainly manual, which has a high technical threshold. Given this reliance on manual labor and the high technical barrier, the inventors of this case conducted research and conceived of a geometric image synthesis scheme based on a text-based graph model. This involves pre-training a text-based graph model, then inputting the user's geometric image request text (described in natural language) into the model, and finally using the model to generate the geometric image.

[0060] Since the geometric image synthesis scheme based on the text-based graph model can synthesize images according to the user's text describing the geometric image requirements in natural language, the technical threshold is relatively low for users. However, the synthesis effect of this scheme is not good, and it is difficult to synthesize geometric images that require accuracy, complex logic and specific relationships.

[0061] Given the numerous shortcomings of geometric image synthesis schemes based on the Wensheng graph model, the inventors of this case conducted further research and, through continuous research, finally proposed a geometric image synthesis method with better results, which overcomes the shortcomings of the aforementioned geometric image synthesis methods.

[0062] The geometric image synthesis method provided in this application will be described below through the following embodiments.

[0063] Please see Figure 1 The diagram illustrates a flowchart of a geometric image synthesis method provided in an embodiment of this application. This geometric image synthesis method may include:

[0064] Step S101: Obtain the user's geometric image request text described in natural language.

[0065] Among them, the geometric image requirement text is the text describing the geometric image requirement in natural language.

[0066] In this embodiment, the geometric image requirement text can be either the geometric image requirement text entered by the user in text input mode, or it can be the text obtained by converting the voice description of the geometric image requirement entered by the user in voice input mode.

[0067] Step S102: Using the code big language model, generate code for synthesizing geometric images based on the geometric image requirement text, and obtain the current image synthesis code.

[0068] Compared to image synthesis schemes based on text-based graph models, this application does not directly generate geometric images from the geometric image requirement text. Instead, it first generates code for synthesizing geometric images based on the geometric image requirement text, and then generates geometric images based on the code.

[0069] For the specific task of geometric image synthesis, using a code-based language model to generate image synthesis code, and then generating geometric images based on this code, is superior to methods that directly generate geometric images from textual requirements in terms of accuracy and debuggability. Specifically, the advantage in accuracy lies in the mathematical precision of the geometric images defined by the code (such as strict side lengths, angles, and coordinates), avoiding the ambiguity and random deviations of directly generated images. The advantage in debuggability lies in the ability to directly locate and modify problems in the code when the results are unsatisfactory.

[0070] Step S103: Synthesize multiple geometric images based on the current image synthesis code.

[0071] After obtaining the current image synthesis code, multiple geometric images are synthesized based on the current image synthesis code.

[0072] It should be noted that errors may occur during the process of synthesizing geometric images based on the current image synthesis code. If an error occurs during the process of synthesizing geometric images based on the current image synthesis code, the error information generated during the synthesis of geometric images will be collected and provided to the code big language model. The code big language model will then modify the current image synthesis code based on the error information.

[0073] Error information generated during the process of synthesizing geometric images based on the current image synthesis code is provided to the code big language model. The code big language model can understand the meaning of the error information, thereby locating the root cause of the problem in the code and generating code that fixes the problem.

[0074] It should be noted that the code big language model cannot guarantee that the image synthesis code will be generated correctly 100% on the first try. The introduction of the above code repair mechanism eliminates the need for manual intervention in debugging, which greatly improves efficiency.

[0075] Step S104: Verify multiple geometric images by combining them with the geometric image requirement text, and determine the degree of conformity of the current image synthesis code with the geometric image requirement text based on the verification results.

[0076] Specifically, multiple geometric images can be verified at both the visual semantic and geometric levels by combining the geometric image requirement text. After obtaining the verification results, the degree of conformity of the current image synthesis code to the geometric image requirement text can be determined based on the verification results.

[0077] Among them, the degree to which the current image synthesis code conforms to the geometric image requirement text represents the extent to which the current image synthesis code conforms to the requirements described in the geometric image requirement text, and it can reflect the performance of the current image synthesis code.

[0078] Specifically, the process of determining the compliance of the current image synthesis code with the geometric image requirement text based on the verification results may include: determining the qualified geometric images (i.e., geometric images that meet the requirements described in the geometric image requirement text) from multiple geometric images based on the verification results, calculating the proportion of the qualified geometric images in the multiple geometric images, and determining the calculated proportion as the compliance of the current image synthesis code with the geometric image requirement text.

[0079] Step S105a: If the conformity of the current image synthesis code to the geometric image requirement text is greater than or equal to the preset conformity threshold, then output the verified geometric image and / or output the current image synthesis code.

[0080] In one possible implementation, only the verified geometric images can be output, so that the user can directly obtain the required geometric images.

[0081] In another possible implementation, only the current image synthesis code (i.e., the image synthesis code with a conformity greater than or equal to a preset conformity threshold) can be output, and the user can generate the required geometric image based on the output image synthesis code.

[0082] In another possible implementation, both the validated geometric image and the current image synthesis code (i.e., the image synthesis code with a conformity greater than or equal to a preset conformity threshold) can be output simultaneously. In some application scenarios, users may require a large number of geometric images, and the number of geometric images directly output may not meet the user's requirements. In this case, more geometric images can be synthesized based on the output image synthesis code.

[0083] In addition, the current image synthesis code (i.e., the image synthesis code with a conformity greater than or equal to the preset conformity threshold) is output to the user, so that the user can read the code and understand how the code achieves the requirements. The user can also modify the code (for example, adjust some details, add some new functions, etc.) to make the code meet the user's personalized needs.

[0084] The geometric image synthesis method provided in this application automates the entire process from understanding geometric image requirements to geometric image synthesis. The synthesis process requires no manual intervention; users only need to describe their geometric image requirements using natural language, eliminating the need for professional programming skills, computer graphics knowledge, or complex drawing software operation skills, thus significantly lowering the technical barrier. Furthermore, the geometric image synthesis method provided in this application significantly improves the efficiency of geometric image synthesis, reducing time and labor costs. In addition, this application first uses a large code language model to generate code for synthesizing geometric images, and then synthesizes geometric images based on this code. This image synthesis strategy has significant advantages in terms of accuracy and scalability. This application also introduces a verification operation for the synthesized image and quantifies the verification result into a compliance degree, making decisions based on the compliance degree to output high-quality results.

[0085] The above embodiments mention that if the conformity of the current image synthesis code to the geometric image requirement text is greater than or equal to a preset conformity threshold, then a verified geometric image is output, and / or the current image synthesis code is output. In practical applications, there are often cases where the conformity of the current image synthesis code to the geometric image requirement text is less than the preset conformity threshold. In view of this, such as Figure 1 As shown, the geometric image synthesis method provided in the above embodiments may further include:

[0086] Step S105b: If the conformity of the current image synthesis code to the geometric image requirement text is less than the preset conformity threshold, then based on the problems existing in the unqualified geometric images among multiple geometric images, generate cognitive-level and enlightening feedback prompts.

[0087] The problem of the failed verification of multiple geometric images was discovered during the verification of multiple geometric images.

[0088] Among them, cognitive-level, inspiring feedback prompts are information that can inspire and guide the code big language model to think more deeply and help the code big language model solve problems. Its core purpose is not to "inform" but to "inspire" and "guide".

[0089] Step S106b: Using the code big language model, modify the current image synthesis code according to the feedback prompts, use the modified code as the current image synthesis code, and execute step S103 and subsequent steps.

[0090] In practical applications, in order to output high-quality results, steps S101 to S104, S105b, and S106b may be executed multiple times. To prevent infinite loops, a maximum number of iterations (e.g., 5 times) can be set. If the maximum number of iterations is reached, the process will end directly.

[0091] In some embodiments of this application, the specific implementation process of the above-mentioned "step S102: using the code big language model to generate code for synthesizing geometric images based on the geometric image requirement text" is described.

[0092] There are multiple ways to generate code for synthesizing geometric images based on the geometric image requirement text using the code big language model. This embodiment provides the following two implementation methods.

[0093] The first implementation method is to use a code-large language model to directly generate code for synthesizing geometric images based on the text requesting the geometric images.

[0094] Specifically, firstly, a first Prompt template pre-configured for the code generation task is obtained. The first Prompt template includes image requirement information slots. The first Prompt template is used to prompt the code big language model to generate code for synthesizing geometric images based on the image requirement information in the image requirement information slots. Then, the geometric image requirement text is filled into the image requirement information slots in the first Prompt template to obtain the first Prompt prompt instruction. Finally, the first Prompt prompt instruction is input into the code big language model to obtain the code for synthesizing geometric images output by the code big language model.

[0095] In this implementation, the code-based large language model is obtained by synthesizing training geometric image requirement text (text described in natural language) and corresponding standard image code, and then fine-tuning the pre-trained large language model.

[0096] The second implementation method is to first convert the geometric image requirement text into a geometric constraint set, and then use the code big language model to generate code for synthesizing geometric images based on the geometric constraint set.

[0097] This implementation does not directly generate code for synthesizing geometric images based on the geometric image requirement text. Instead, it first converts the geometric image requirement text into a set of geometric constraints, and then uses the code big language model to generate code for synthesizing geometric images based on the set of geometric constraints.

[0098] For example, the geometric image requirement text is "Generate 10 equilateral triangles with a side length of 5cm, evenly distributed within a circular area with a diameter of 20cm". The geometric constraint set obtained by converting this geometric image requirement text is {number of entities = 10; type = equilateral triangle; side length = 5cm; ...}.

[0099] In one possible implementation, the process of converting geometric image requirement text into a set of geometric constraints may include: converting geometric image requirement text into a set of geometric constraints based on a pre-built geometric terminology mapping library.

[0100] The geometric terminology mapping library includes various geometric shape types and the attributes of each geometric shape type (such as side length, angle, etc.). It should be noted that geometric shape types can include coarse-grained shape types (such as triangles, circles, trapezoids, etc.) and fine-grained shape types under coarse-grained shape types (such as equilateral triangles, isosceles triangles, right triangles, etc. under triangles). Furthermore, various geometric shape types can include independent shape types and composite shape types.

[0101] Specifically, based on a pre-built geometric terminology mapping library, the process of converting geometric image requirement text into a geometric constraint set may include: extracting the target geometric shape type from the geometric image requirement text; obtaining the attributes of the target geometric shape type from the geometric terminology mapping library; extracting the attribute values ​​corresponding to the attributes of the target geometric shape type from the geometric image requirement text; and constructing a geometric constraint set based on the target geometric shape type, the attributes of the target geometric shape type, and the attribute values ​​corresponding to the attributes of the target geometric shape type.

[0102] After converting the geometric image requirement text into a geometric constraint set, the Code Big Language Model is used to generate code for synthesizing the geometric image based on the geometric constraint set. Specifically, firstly, a second Prompt template pre-configured for the code generation task is obtained. The second Prompt template includes geometric constraint information slots. The second Prompt template is used to prompt the Code Big Language Model to generate code for synthesizing the geometric image based on the geometric constraint information in the geometric constraint information slots. Then, the geometric constraint set is filled into the geometric constraint information slots in the second Prompt template to obtain the second Prompt prompt instruction. Finally, the second Prompt prompt instruction is input into the Code Big Language Model to obtain the code for synthesizing the geometric image output by the Code Big Language Model.

[0103] In this implementation, the code-based large language model is obtained by fine-tuning the pre-trained large language model using a training geometric constraint set (derived from the training geometric image requirement text) and the corresponding standard image synthesis code.

[0104] This implementation effectively decouples the complex tasks of natural language understanding and code generation by introducing a structured intermediate representation (i.e., a set of geometric constraints), bringing significant benefits: First, it forces the mapping of ambiguous and diverse natural language descriptions to precise constraint types and parameters, greatly reducing ambiguity. Second, for the Code Big Language Model, generating code from a structured set of geometric constraints is far simpler than parsing free text from scratch, providing an extremely clear "requirements document" that the model can simply act as a "skilled programmer" to implement. Third, because the input is a structured set of geometric constraints, the randomness and error rate of the Code Big Language Model when generating code are significantly reduced; that is, the introduction of the geometric constraint set can effectively improve the accuracy of code generation.

[0105] By utilizing the code big language model, code for synthesizing geometric images is generated based on the geometric image requirement text. After obtaining the current image synthesis code, multiple geometric images are generated based on the current image synthesis code. Considering that the geometric images generated based on the current image synthesis code may not conform to the geometric image requirement text, the multiple geometric images can be verified in conjunction with the geometric image requirement text. In some embodiments of this application, the specific implementation process of "verifying multiple geometric images in conjunction with the geometric image requirement text" in step S104 is described.

[0106] In one possible implementation, such as Figure 2 As shown, the process of verifying multiple geometric images in conjunction with the geometric image requirement text can include:

[0107] Step S201: Using a visual large language model, combined with the text of geometric image requirements, perform visual semantic verification on multiple geometric images respectively.

[0108] Visual semantic verification utilizes the cognitive understanding capabilities of a visual large language model to interpret and judge the content of geometric images in a manner similar to human "picture description." It does not focus on geometric parameters accurate to the pixel level, but rather on whether visual concepts and relationships are correct. In other words, it performs overall semantic conformity verification on geometric images. Its core task is to answer the question of whether the image, in terms of meaning, conforms to the requirements described in the geometric image requirement text.

[0109] In one possible implementation, visual semantic verification may include semantic and relational verification as well as quantitative verification, wherein semantic and relational verification includes visual entity semantic verification and spatial relational verification, and quantitative verification includes quantity conformity verification.

[0110] That is, for each of the multiple geometric images, a visual big language model is used, combined with the geometric image requirement text, to perform visual entity semantic verification, spatial relationship verification, and quantity conformity verification on the geometric image.

[0111] Visual entity semantic verification is used to verify whether the types and attributes of independent visual entities in a geometric image conform to the requirements described in the geometric image requirement text. For example, if the requirement text describes a geometric image as "a triangle and a circle", then visual entity semantic verification verifies whether there is indeed a visual entity identified as a "triangle" and a visual entity identified as a "circle" in the geometric image. As another example, if the requirement text describes a geometric image as "a red circle and a blue square", then visual entity semantic verification verifies whether there are indeed visual entities identified as a "circle" and a visual entity identified as a "square" in the geometric image. If there are "circles" and "squares", it is also necessary to verify whether the "circle" is red and whether the "square" is blue.

[0112] Spatial relationship verification is used to verify whether the spatial relationships (relative positions and topological relationships between different visual entities) in a geometric image meet the requirements described in the geometric image requirements text. For example, if the requirements text describes "a circle is inside a square", then spatial relationship verification verifies whether, visually, the circle is located inside the square (i.e., an inclusion relationship). As another example, if the requirements text describes "a triangle is on the left side of a rectangle", then spatial relationship verification verifies whether, from the perspective of the overall composition, the triangle is on the left side of the rectangle.

[0113] Quantity compliance verification is used to verify whether the number of various visual entities in a geometric image conforms to the requirements described in the geometric image requirements document. For example, if the requirements document describes "five triangles of different sizes," then quantity compliance verification will check whether there are exactly five visual entities in the geometric image that are identified as "triangles."

[0114] When using a visual big language model and combining geometric image requirement text to perform visual semantic verification on any geometric image, the first step is to obtain a pre-configured Prompt template for the image verification task. This Prompt template includes image information slots and image requirement information slots. The Prompt template is used to prompt the visual big language model to perform semantic verification of individual visual entities, spatial relationship verification of different visual entities, and quantity conformity verification of various visual entities in the image information slots. Then, the geometric image is filled into the image information slots, and the geometric image requirement text is filled into the image requirement information slots to obtain the Prompt prompt instruction. The Prompt prompt instruction is input into the visual big language model to obtain the verification result output by the visual big language model.

[0115] Step S202: Combine the geometric image requirement text to perform geometric verification on each geometric image that has passed the semantic verification.

[0116] In this embodiment, geometric verification includes precise parameterization verification.

[0117] Specifically, in conjunction with the requirements for geometric image synthesis, the process of performing geometric level verification on each geometric image that has passed semantic level verification can include: for each geometric image that has passed semantic level verification, determining the core geometric parameters (such as size, position, angle, etc.) and target parameter values ​​of the core geometric parameters based on the geometric image requirement text; obtaining the actual parameter values ​​of the core geometric parameters by measuring the visual entities in the geometric image based on computer vision algorithms (such as subpixel edge detection, Hough transform, etc.); and performing precise compliance verification of the core geometric parameters on the geometric image based on the actual parameter values ​​and target parameter values ​​of the core geometric parameters.

[0118] When determining the core geometric parameters and their target values, one possible implementation is to directly analyze the geometric image requirement text to determine the core geometric parameters and their target values. Another possible implementation is to determine the core geometric parameters and their target values ​​based on the geometric constraint set obtained by transforming the geometric image requirement text.

[0119] When performing precise conformity verification of the core geometric parameters on the geometric image based on the actual and target values ​​of the core geometric parameters, the deviation between the actual and target values ​​can be calculated. If the deviation is less than a preset deviation threshold, the geometric image is determined to have passed the precise conformity verification, meaning it passes the geometric level verification. It should be noted that there may be more than one core geometric parameter. If there are multiple core geometric parameters, each with an actual and a target value, the difference between the actual and target values ​​of each core parameter is calculated. If all deviations are less than the corresponding preset deviation threshold, the geometric image is determined to have passed the precise conformity verification.

[0120] This application combines human visual cognitive logic (when humans view images, they first perceive the overall structure and the relationships between elements, rather than individual pixels or absolute coordinates) with precise geometric calculations to construct a progressive quality assessment system from macroscopic semantic understanding to microscopic parameter verification. This system enables comprehensive and reliable control over the quality of geometric image synthesis. This verification mechanism ensures both semantic compliance and precise geometric compliance, significantly improving the reliability and accuracy of automated image synthesis.

[0121] It should be noted that this embodiment does not limit the verification of multiple geometric images at the visual semantic level first, and then the verification of geometric images that pass the visual semantic level verification at the geometric level. In another possible implementation, multiple geometric images can be verified at the visual semantic level and at the geometric level in parallel. For example, if there are 10 geometric images, the visual semantic level verification can be performed on all 10 geometric images, and the geometric level verification can be performed on all 10 geometric images at the same time.

[0122] After verification, the conformity of the current image synthesis code with the geometric image requirement text can be determined based on the verification results. Specifically, firstly, geometric images that conform to the requirements described in the geometric image requirement text are identified based on the verification results. Specifically, if a geometric image passes both semantic and geometric verification, it can be determined that the geometric image conforms to the requirements described in the geometric image requirement text. After identifying the geometric images that conform to the requirements described in the geometric image requirement text, the proportion of geometric images that conform to the requirements described in the geometric image requirement text among multiple geometric images is calculated. The calculated proportion is used as the conformity of the current image synthesis code with the geometric image requirement text.

[0123] After obtaining the conformity of the current image synthesis code with the geometric image requirement text, it can be determined whether the conformity of the current image synthesis code with the geometric image requirement text is greater than or equal to a preset conformity threshold (e.g., 95%). If the conformity of the current image synthesis code with the geometric image requirement text is greater than or equal to the preset conformity threshold, a qualified geometric image can be output, and the current image synthesis code (i.e., the image synthesis code with a conformity greater than or equal to the preset conformity threshold) can also be output.

[0124] If the current image synthesis code's compliance with the geometric image requirement text is less than a preset compliance threshold, then based on the problems found in the unqualified geometric images among multiple geometric images, cognitive-level and heuristic feedback information is generated. Subsequently, the code big language model can be used to modify the current image synthesis code according to the feedback information, and the modified code is used as the current image synthesis code to re-execute geometric image generation, geometric image verification, and other operations until the termination condition is met.

[0125] Below are two examples of feedback prompts: (1) Feedback prompt involving logic correction: "The visual big language model confirms that the number of triangles is 8, which does not meet the requirement of 10. Please check the loop logic in the code to ensure that the number of iterations is correct."; (2) Feedback prompt involving distribution optimization: "The visual big language model believes that the triangles in the current image 'visually cluster in the upper left corner and do not achieve uniform distribution'. It is recommended to modify the coordinate generation algorithm, and try using Poisson disk sampling or golden angle distribution methods to improve visual uniformity."

[0126] In one possible implementation, the feedback information may include problems with the geometric image and heuristic guidance on solving those problems.

[0127] When generating cognitive-level, insightful feedback prompts based on the problems found in non-compliant geometric images from multiple geometric images, one possible implementation is to generate a comprehensive evaluation report based on the verification results of multiple geometric images and the problems discovered during the verification process, and then generate cognitive-level, insightful feedback prompts based on the comprehensive evaluation report.

[0128] The comprehensive evaluation report may include the problems found in the non-compliant geometric images (such as problems at the semantic level and / or at the geometric level), as well as the verification results indicating whether the geometric image meets the requirements (i.e., the requirements described in the geometric image requirement text), the percentage of qualified geometric images (i.e., the aforementioned compliance rate), and the statistical analysis results of the problems found in the non-compliant geometric images, etc.

[0129] In one possible implementation, a comprehensive evaluation report can be directly generated based on the verification results corresponding to multiple geometric images and the image problems found during the verification process. In another possible implementation, an evaluation report corresponding to each geometric image can be generated first based on the verification results corresponding to each geometric image and the image problems found during the verification process (some images may not have problems), and then a comprehensive evaluation report can be generated based on the evaluation reports corresponding to multiple geometric images.

[0130] The geometric image synthesis method provided in this application has the following beneficial effects:

[0131] (1) Full-process automation and low threshold: It realizes full-process automation from natural language requirement understanding to final image synthesis, without human intervention. Users only need to provide geometric image requirements described in natural language, without having programming or professional drawing skills, which greatly improves ease of use.

[0132] (2) High accuracy and high reliability of the generated results: By using the code big language model to generate executable code, the fuzzy natural language instructions are transformed into precise and structured computer instructions, ensuring that the generated geometric images strictly conform to the user's intention in terms of size, quantity, spatial relationship and other attributes, overcoming the randomness and fuzziness of traditional text-based image models.

[0133] (3) Reusability and interpretability of output results: The final output is not only a geometric image, but also a readable, modifiable and reusable image synthesis code. This ensures the repeatability of the generated results and gives users the interpretability of the generation process and the ability to flexibly re-edit the generated results, which facilitates integration, auditing and iteration.

[0134] (4) An image verification system that integrates semantic cognition and geometric precision. In the verification process, in addition to verifying the accuracy of geometric parameters, the cognitive ability of the visual language big model is introduced into the verification process of geometric images. This not only ensures engineering-level geometric precision, but also ensures that the overall semantics meet the requirements.

[0135] (5) Built-in intelligent quality inspection and continuous optimization capabilities: Through the verification feedback mechanism, it can automatically identify the gap between the generated results and the requirements, and drive the code to perform iterative optimization, ensuring the stability and reliability of the output quality, and enabling the solution to have the intelligence of self-correction and continuous improvement.

[0136] This application also provides a geometric image synthesis system, such as... Figure 3 As shown, the geometric image synthesis system may include: a requirement definition unit 301, a code generation unit 302, an image synthesis unit 303, an image verification unit 304, and a decision output unit 305.

[0137] The requirement definition unit 301 is used to obtain the geometric image requirement text described by the user in natural language.

[0138] The code generation unit 302 is used to generate code for compositing geometric images based on the geometric image requirement text using a code big language model, thereby obtaining the current image synthesis code.

[0139] Image synthesis unit 303 is used to synthesize multiple geometric images based on the current image synthesis code.

[0140] Image verification unit 304 is used to verify multiple geometric images in conjunction with geometric image requirement text, and determine the degree of conformity of the current image synthesis code to the geometric image requirement text based on the verification results.

[0141] The decision output unit 305 is used to output a qualified geometric image when the compliance degree is greater than or equal to a preset compliance degree threshold, and / or output the current image synthesis code so as to directly synthesize the geometric image based on the output code.

[0142] In one possible implementation, the geometric image generation system may further include a feedback prompt information generation unit 306.

[0143] The feedback prompt information generation unit 306 is used to generate cognitive-level, heuristic feedback prompt information based on the problems existing in the unqualified geometric images among the multiple geometric images when the compliance degree is less than the preset compliance degree threshold, wherein the problems are discovered when verifying the multiple geometric images.

[0144] The code generation unit 302 is also used to modify the current geometric image synthesis code based on the feedback prompt information using the code language model.

[0145] In one possible implementation, when the code generation unit 302 generates code for synthesizing the geometric image based on the geometric image requirement text using a code-large language model, it specifically performs the following:

[0146] Convert the geometric image requirement text into a set of geometric constraints;

[0147] Using a code-large language model, code for synthesizing geometric images is generated based on the set of geometric constraints.

[0148] In one possible implementation, the code generation unit 302, when converting the geometric image requirement text into a set of geometric constraints, specifically performs the following:

[0149] Based on a pre-built geometric terminology mapping library, the geometric image requirement text is converted into a set of geometric constraints, wherein the geometric terminology mapping library includes multiple geometric shape types and attributes of each geometric shape type.

[0150] In one possible implementation, when the code generation unit 302 converts the geometric image requirement text into a set of geometric constraints based on a pre-built geometric terminology mapping library, it is specifically used for:

[0151] Extract the target geometric shape type from the geometric image requirement text;

[0152] Obtain the attributes of the target geometry type from the geometry terminology mapping library;

[0153] Extract the attribute values ​​corresponding to the attributes of the target geometric shape type from the geometric image requirement text;

[0154] A set of geometric constraints is constructed based on the target geometric shape type, the attributes of the target geometric shape type, and the attribute values ​​corresponding to the attributes of the target geometric shape type.

[0155] In one possible implementation, when verifying the multiple geometric images in conjunction with the geometric image requirement text, the image verification unit 304 is specifically used for:

[0156] Using a visual large language model, combined with the geometric image requirement text, semantic verification is performed on the multiple geometric images respectively;

[0157] Based on the geometric image requirement text, a geometric verification is performed on each geometric image that passes the semantic verification.

[0158] In one possible implementation, when the image verification unit 304 performs semantic-level verification on the multiple geometric images using a visual large language model in conjunction with the geometric image requirement text, it specifically performs the following:

[0159] For each of the multiple geometric images, a visual big language model is used, combined with the geometric image requirement text, to perform visual entity semantic verification, spatial relationship verification, and quantity conformity verification on the geometric image.

[0160] Specifically, the visual entity semantic verification is used to verify whether the types and attributes of independent visual entities in the geometric image meet the requirements described in the geometric image requirement text; the spatial relationship verification is used to verify whether the spatial relationships between different visual entities in the geometric image meet the requirements described in the geometric image requirement text; and the quantity compliance verification is used to verify whether the quantity of each type of visual entity in the geometric image meets the requirements described in the geometric image requirement text.

[0161] In one possible implementation, when the image verification unit 304 performs geometric level verification on each geometric image that has passed semantic level verification in conjunction with the geometric image requirement text, it is specifically used to:

[0162] For each geometric image that passes semantic-level verification:

[0163] Based on the geometric image requirement text, determine the core geometric parameters and the target parameter values ​​of the core geometric parameters;

[0164] The actual parameter values ​​of the core geometric parameters are obtained by measuring the visual entities in the geometric image;

[0165] Based on the actual parameter values ​​and target parameter values ​​of the core geometric parameters, the geometric image is subjected to a precise conformity verification of the core geometric parameters.

[0166] In one possible implementation, when determining the conformity of the current image synthesis code to the geometric image requirement text based on the verification result, the image verification unit 304 is specifically used for:

[0167] Geometric images that pass both semantic and geometric level verification are identified as geometric images that meet the requirements described in the geometric image requirement text.

[0168] The percentage of geometric images that meet the requirements described in the geometric image requirement text among the multiple geometric images is used as the degree of compliance of the current image synthesis code with the geometric image requirement text.

[0169] In one possible implementation, the code generation unit 302 is also used for:

[0170] If an error occurs in the process of generating a geometric image based on the current image synthesis code, the image synthesis unit 303 uses the code big language model to modify the current image synthesis code according to the error information generated during the generation of the geometric image, and provides the modified code as the current image synthesis code to the image synthesis unit 303 for image synthesis.

[0171] The geometric image synthesis system provided in this application embodiment automates the entire process from understanding geometric image requirements to geometric image synthesis. The synthesis process requires no manual intervention. For users, since they only need to describe their geometric image requirements using natural language, no professional programming skills, graphics knowledge, or complex drawing software operation skills are required, significantly lowering the technical threshold. Simultaneously, the geometric image synthesis system provided in this application embodiment significantly improves the efficiency of geometric image synthesis, reducing time and labor costs. Furthermore, the geometric image synthesis system provided in this application embodiment first uses a large code language model to generate code for synthesizing geometric images, and then synthesizes geometric images based on this code. This image synthesis strategy has significant advantages in terms of accuracy and scalability. The geometric image synthesis system provided in this application embodiment introduces a large visual language model into the image verification stage. In addition to verifying the precise conformity of geometric parameters, it also utilizes the cognitive understanding ability of the large visual language model to verify the overall semantic conformity of the geometric images. This not only ensures engineering-level geometric accuracy but also guarantees overall semantic conformity.

[0172] This application also provides an electronic device, which may include at least one processor and a memory connected to the processor.

[0173] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application; the memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage.

[0174] The memory is used to store computer programs, and the processor is used to execute the computer programs so that the electronic device can implement the geometric image synthesis method provided in the above embodiments.

[0175] This application also provides a computer storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device is able to implement the geometric image synthesis method provided in the above embodiments.

[0176] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the geometric image synthesis method provided in the above embodiments.

[0177] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0179] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0180] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method of geometric image synthesis, characterized by, The method comprises the following steps: obtaining a user's natural language description of a geometric image requirement text; using a code large language model to generate code for synthesizing a geometric image according to the geometric image requirement text, obtaining current image synthesis code; synthesizing multiple geometric images based on the current image synthesis code; verifying the multiple geometric images in combination with the geometric image requirement text, and determining the degree of conformity of the current image synthesis code to the geometric image requirement text according to the verification result; if the degree of conformity is greater than or equal to a preset conformity threshold, outputting the verified geometric image, and / or outputting the current image synthesis code for directly synthesizing a geometric image based on the output code.

2. The geometry image synthesis method according to claim 1, wherein, Further comprising: if the degree of conformity is less than the preset conformity threshold, generating cognitive-level and heuristic feedback prompt information according to the problems existing in the geometric image that fails the verification, wherein the problems are found when verifying the multiple geometric images; using the code large language model to modify the current image synthesis code according to the feedback prompt information; using the modified code as the current image synthesis code to execute the step of synthesizing multiple geometric images based on the current image synthesis code and subsequent steps.

3. The geometry image synthesis method according to claim 1 or 2, characterized in that, The step of using the code large language model to generate code for synthesizing a geometric image according to the geometric image requirement text comprises the following steps: converting the geometric image requirement text into a geometric constraint set; using the code large language model to generate code for synthesizing a geometric image according to the geometric constraint set.

4. The geometry image synthesis method according to claim 3, wherein The step of converting the geometric image requirement text into a geometric constraint set comprises the following steps: based on a pre-constructed geometric term mapping library, converting the geometric image requirement text into a geometric constraint set, wherein the geometric term mapping library includes multiple geometric figure types and attributes of each geometric figure type.

5. The geometry image synthesis method according to claim 4, wherein The step of converting the geometric image requirement text into a geometric constraint set based on the pre-constructed geometric term mapping library comprises the following steps: extracting a target geometric figure type from the geometric image requirement text; obtaining the attributes of the target geometric figure type from the geometric term mapping library; extracting attribute values corresponding to the attributes of the target geometric figure type from the geometric image requirement text; constructing a geometric constraint set according to the target geometric figure type, the attributes of the target geometric figure type, and the attribute values corresponding to the attributes of the target geometric figure type.

6. The geometry image synthesis method according to claim 1 or 2, characterized by, The step of verifying the multiple geometric images in combination with the geometric image requirement text comprises the following steps: using a visual large language model to verify the multiple geometric images respectively at a semantic level in combination with the geometric image requirement text; verifying each geometric image that passes the semantic level verification at a geometric level in combination with the geometric image requirement text.

7. The geometry image synthesis method according to claim 6, wherein, The step of using a visual large language model to verify the multiple geometric images respectively at a semantic level in combination with the geometric image requirement text comprises the following steps: For each of the plurality of geometric images, a visual large language model is used in combination with the geometric image requirement text to perform visual entity semantic verification, spatial relationship verification, and quantity compliance verification on the geometric image. The visual entity semantic verification is used to verify whether the types and attributes of independent visual entities in the geometric image meet the requirements described in the geometric image requirement text, the spatial relationship verification is used to verify whether the spatial relationships between different visual entities in the geometric image meet the requirements described in the geometric image requirement text, and the quantity compliance verification is used to verify whether the quantities of visual entities of each type in the geometric image meet the requirements described in the geometric image requirement text.

8. The geometry image synthesis method according to claim 6, wherein, The geometric level verification on each geometric image that passes the semantic level verification in combination with the geometric image requirement text includes: For each geometric image that passes the semantic level verification: According to the geometric image requirement text, determine the core geometric parameter and the target parameter value of the core geometric parameter; By measuring the visual entities in the geometric image, obtain the actual parameter value of the core geometric parameter; According to the actual parameter value of the core geometric parameter and the target parameter value of the core geometric parameter, perform precise compliance verification of the core geometric parameter on the geometric image.

9. The geometry image synthesis method according to claim 6, wherein, The determination of the compliance degree of the current image synthesis code with respect to the geometric image requirement text according to the verification result includes: Determine the geometric image that passes the semantic level verification and the geometric level verification as a geometric image that meets the requirements described in the geometric image requirement text; Statistically determine the proportion of the geometric image that meets the requirements described in the geometric image requirement text in the plurality of geometric images as the compliance degree of the current image synthesis code with respect to the geometric image requirement text.

10. The geometry image synthesis method according to claim 1 or 2, characterized by, Further comprising: If an error occurs in the process of generating a geometric image based on the current image synthesis code, use the code large language model to modify the current image synthesis code according to the error information generated in the process of generating the geometric image.

11. An electronic device, comprising: Comprise at least one processor and a memory connected with the processor, wherein: The memory is used to store a computer program; The processor is used to execute the computer program, so that the electronic device can implement the geometric image synthesis method according to any one of claims 1-10.

12. A computer storage medium, characterized in that The storage medium carries one or more computer programs, which can make the electronic device implement the geometric image synthesis method according to any one of claims 1-10 when the one or more computer programs are executed by the electronic device.

13. A geometry image generation system, characterized by, Comprise: A requirement definition unit, a code generation unit, an image synthesis unit, an image verification unit, and a decision output unit; The requirement definition unit is used to obtain a geometric image requirement text described in natural language by a user; The code generation unit is used to generate a code for synthesizing a geometric image by using a code large language model according to the geometric image requirement text, to obtain a current image synthesis code; The image synthesis unit is used to synthesize a plurality of geometric images based on the current image synthesis code; The image verification unit is configured to verify the plurality of geometric images in combination with the geometric image requirement text, and determine a compliance degree of a current image synthesis code to the geometric image requirement text according to a verification result. The decision output unit is configured to output a verified geometric image when the compliance degree is greater than or equal to a preset compliance threshold, and / or output the current image synthesis code, so as to synthesize the geometric image based on the output code directly.

Images