A structured image generation method, apparatus, device and medium

Abstract

The application discloses a structured image generation method, device, equipment and medium, and relates to the technical field of image generation. The method comprises the following steps: obtaining natural language input of a structured image to be generated; performing semantic analysis on the natural language input to obtain a formal specification and a constraint set; generating an intermediate symbolic representation in a constraint-guided manner according to the formal specification and the constraint set; obtaining constraint violation energy and error information through a symbolic verifier according to the intermediate symbolic representation and the constraint set; obtaining a legal intermediate symbolic representation according to the constraint violation energy and the error information; and obtaining a structured image through a deterministic rendering mapping according to the legal intermediate symbolic representation. The method can accurately generate a structured image.

Description

Technical Field

[0001] This application relates to the field of image generation technology, and in particular to a method, apparatus, device and medium for generating structured images. Background Technology

[0002] Structured image generation is an important intersection of artificial intelligence and computer graphics, widely used in professional scenarios such as drawing chemical molecular structures and visualizing scientific data. Unlike natural images, structured images have strict requirements for topological validity, numerical consistency, and editability: the number of atomic valence bonds in a molecular diagram must conform to chemical rules, and the coordinates of data points in a graph must strictly match the coordinate axis scale. These constraints are essentially formally verifiable structural problems, rather than problems that can be determined solely by visual perception.

[0003] Existing technologies can be mainly divided into three categories. The first category is the diffusion-based text-to-graph model, which generates images by learning the distribution of text to pixels, but lacks explicit modeling of structural objects, making it difficult to guarantee the stable satisfaction of hard constraints such as valence rules and coordinate mapping. The second category is procedural plotting practices, which generate Matplotlib code or SMILES and then execute rendering, but lacks a systematic closed loop between the generator and the validator, making it impossible to automatically locate and correct errors after they occur. The third category is constraint decoding technology, which controls token generation through syntax constraints, but its constraints are mostly limited to the string level and are difficult to cover topological and domain rules that require structural operations for determination.

[0004] Therefore, existing technologies cannot accurately generate structured images. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for generating structured images, which can accurately generate structured images.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application provides a structured image generation method, including: Obtain the natural language input for the structured image to be generated; Based on the natural language input, semantic parsing is performed to obtain a set of formal specifications and constraints; Based on the formal specifications and constraint set, intermediate symbol representations are generated using a constraint-guided approach. Based on the intermediate symbolic representation and the constraint set, the constraint violation energy and error information are obtained through the symbolic verifier. Based on the constraint violation energy and error information, a valid intermediate symbol representation is obtained; Based on the legal intermediate symbols, a structured image is obtained through deterministic rendering mapping.

[0007] Optionally, obtaining a valid intermediate symbol representation based on constraint violation energy and error information includes: When the constraint violates the energy threshold, the current intermediate symbol representation is taken as a valid intermediate symbol representation. When the energy of the constraint violation is greater than the energy threshold, if the current iteration round has not reached the maximum round, an error correction prompt is generated based on the error information, and an intermediate symbol representation is regenerated based on the error correction prompt and the formal specification. If the current iteration reaches the maximum iteration, select the intermediate symbol representation with the lowest constraint violation energy from the multiple generated intermediate symbol representations as the legal intermediate symbol representation.

[0008] Optionally, the energy threshold is obtained in the following way: The energy threshold is determined based on the current iteration round, the historical best energy value, and the convergence parameters.

[0009] Optionally, the step of obtaining constraint violation energy and error information through a symbolic verifier based on intermediate symbolic representations and constraint sets includes: The intermediate symbol representation is parsed, and the parsing result is mapped to a verifiable structural kernel; Based on the set of constraints, the structural kernel is constrained to obtain constraint violation energy and error information.

[0010] Optionally, the step of performing semantic parsing based on natural language input to obtain a set of formal specifications and constraints includes: Based on the natural language input, determine the carrier type and task domain represented by the intermediate symbols; Based on the carrier type and task domain represented by the intermediate symbols, extract the object set, relation set, and attribute set; Based on the set of objects, the set of relations, and the set of attributes, a formal specification is generated, which includes the structure and constraint intent. The structure and constraint intent are compiled into a constraint set, which includes hard constraints and soft constraints.

[0011] Optionally, the step of generating intermediate symbolic representations using a constraint-guided approach based on formal specifications and constraint sets includes: Based on formal specifications, establish the autoregressive probability distribution of the symbol sequence; Based on the hard constraints in the constraint set, a feasible symbol set is selected from the vocabulary; The autoregressive probability distribution is renormalized based on the feasible symbol set; Based on the renormalized probability distribution, the intermediate symbol representation is obtained through decoding.

[0012] Optionally, obtaining a structured image through deterministic rendering mapping based on valid intermediate symbol representation includes: Invoke the deterministic renderer to render the legal intermediate symbol representation into a structured image; Output the structured image.

[0013] Secondly, this application provides a structured image generation apparatus, comprising: The acquisition module is used to acquire the natural language input for the structured image to be generated; The processing module is used to perform semantic parsing on natural language input to obtain a formal specification and constraint set; generate intermediate symbolic representations using a constraint-guided approach based on the formal specification and constraint set; obtain constraint violation energy and error information through a symbolic verifier based on the intermediate symbolic representations and constraint set; and obtain valid intermediate symbolic representations based on the constraint violation energy and error information. The determination module is used to obtain a structured image through deterministic rendering mapping based on the legal intermediate symbol representation.

[0014] Thirdly, this application provides a computing device, including a memory and a processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computing device performs the method as described in any one of the first aspects.

[0015] Fourthly, this application provides a computer-readable storage medium for storing a computer program for performing the method as described in any one of the first aspects.

[0016] As can be seen from the above technical solution, this application has at least the following beneficial effects: In this application, an intermediate symbolic representation is introduced as a bridge between generation and rendering, elevating the generation target from pixel space to symbolic space. At the same time, a symbolic verifier performs strict constraint verification on the intermediate symbolic representation, calculates the constraint violation energy and outputs localizable error information, ensuring that the final output structured image strictly satisfies the domain rules in terms of topological structure and numerical relationships, fundamentally solving the problem of visual similarity but structural errors in the prior art.

[0017] Furthermore, on the one hand, a constraint-guided approach is adopted in the generation phase. Feasible symbol sets are selected from the vocabulary based on hard constraints, and the probability distribution is renormalized, eliminating illegal branches at the token level in advance. On the other hand, in the verification phase, a symbol verifier outputs structured error information. When the constraint violation energy exceeds the energy threshold, error correction prompts are generated based on the error information, driving the generator to regenerate, forming an iterative closed loop of generation, verification, feedback, and correction. This dual verification mechanism enables the system to automatically locate the cause of errors and continuously optimize until a valid intermediate symbol representation is obtained, improving the success rate and stability of generation.

[0018] Finally, this application introduces an energy threshold control mechanism in the feedback loop. When the constraint violation energy is less than or equal to the energy threshold, the current result is directly accepted to avoid unnecessary iterations. When the threshold is exceeded, targeted corrections are made based on the error information. At the same time, the energy threshold is dynamically determined based on the current iteration round, the historical best energy value, and the convergence parameters, so that the system can achieve a balance between generation quality and convergence efficiency, which not only ensures the legality of the final output, but also avoids the waste of computing resources caused by infinite retries.

[0019] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a structured image generation method provided in this application embodiment; Figure 2 A schematic diagram of a structured image generation apparatus provided in an embodiment of this application; Figure 3 This is a schematic diagram of a computing device provided in an embodiment of this application. Detailed Implementation

[0021] The terms "first," "second," and "third," etc., used in this application specification and accompanying drawings are used to distinguish different objects, not to limit a specific order.

[0022] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] To ensure clarity and conciseness in the description of the following embodiments, a brief introduction to the related technologies is given first: Structured images refer to image types with clearly defined topological relationships, numerical mappings, and formally verifiable rules. Typical examples include chemical molecular structure diagrams, statistical charts, flowcharts, and circuit diagrams. Unlike natural images, the correctness of structured images lies not only in their visual rationality and aesthetics, but also in the strict adherence of their structural and numerical relationships to domain rules. For example, in a chemical molecular diagram, the number of connections (valence bonds) of each atom must conform to chemical rules; in a statistical chart, the coordinates of data points must strictly align with the coordinate axis scale and bar; and in a flowchart, edges must connect to nodes and not be suspended. These constraints essentially pertain to formally verifiable discrete structure and topological consistency problems, rather than being determined solely by visual perception.

[0024] Existing structured image generation techniques mainly suffer from the following problems and their causes: First, the diffusion-based text-to-image model directly maps text to pixel images, and its generation process involves conditional probability fitting to pixel space. Because this paradigm lacks explicit modeling of structural objects such as atoms, bonds, coordinate axes, and data points, and struggles to incorporate hard constraints like valence bond rules and coordinate mapping into the generation process, topological illusions frequently occur in the generated results. Molecular diagrams may contain invalid bond numbers or dangling bonds, and graphs may show mismatches between data points and coordinate axes. These errors are often difficult to detect quickly with the naked eye, yet they directly render the output unusable in scientific research and engineering scenarios.

[0025] Secondly, while procedural plotting practices (such as generating Matplotlib code or SMILES before rendering) have recognized the engineering value of generating executable symbols over generating pixels, they lack a systematic closed loop between the generator and the verifier. When the generated code contains syntax errors, runtime errors, or structural errors, the common handling methods are simply retrying or manual correction, lacking a unified constraint representation, energy measurement, and automatic feedback correction mechanism. After an error occurs, the cause cannot be automatically located and the correction cannot be driven.

[0026] Third, although constraint decoding technology can control token generation through syntactic constraints when generating JSON (JavaScript Object Notation) or DSL (Domain-Specific Language), its constraints are mostly limited to the string or syntactic level (such as bracket closure and missing fields), and it is difficult to cover topological and domain rule-level constraints that require structural operations to determine, such as the valence key rules of the subgraph and the numerical consistency of the graph.

[0027] In summary, existing technologies cannot accurately generate structured images that satisfy topological and numerical constraints. The fundamental reason is that the generation process lacks explicit modeling and verifiable intermediate representations of the structural objects; generation and verification are disconnected; constraints fail to cover the topological and domain rule levels; and the output is merely an uneditable pixel image. Therefore, there is an urgent need for a structured image generation method that can move the generation process from pixel space to symbolic space, introduce computable constraint verification and closed-loop correction mechanisms, and output editable structured products.

[0028] In view of this, embodiments of this application provide a structured image generation method, which can be executed by a processing device. The processing device can be a terminal or a server. Terminals include, but are not limited to, smartphones, tablets, laptops, personal digital assistants, or smart wearable devices. The server can be a cloud server, such as a central server in a central cloud computing cluster or an edge server in an edge cloud computing cluster. Alternatively, the server can be a server in a local data center. A local data center refers to a data center directly controlled by the user.

[0029] To address the problems in existing structured image generation technologies, such as frequent topological illusions, inability to automatically correct errors, and uneditable output due to the lack of explicit modeling of structural objects, the disconnect between generation and verification, and the inability of constraints to cover the topological and domain rule levels, this application first transforms natural language input into a formalized specification and a computable set of constraints through semantic parsing. Then, it uses a constraint-guided approach to generate intermediate symbolic representations, preemptively eliminating illegal branches that violate hard constraints at the token level, achieving neural-side verification. Next, a symbolic verifier performs rigorous topological and numerical constraint verification on the intermediate symbolic representations, calculating constraint violation energy and outputting localizable error information, achieving symbolic-side verification. When the constraint violation energy exceeds a threshold, error correction prompts are generated based on the error information, driving the generator to regenerate, forming an iterative correction loop. Finally, deterministic rendering mapping renders the valid intermediate symbolic representations into structured images, outputting the intermediate symbolic representations and verification reports, achieving editable and auditable engineering delivery.

[0030] To make the technical solution of this application clearer and easier to understand, a structured image generation method provided by an embodiment of this application will be described below with reference to the accompanying drawings. Figure 1 The figure shown is a flowchart of a structured image generation method provided in an embodiment of this application. The method includes: S201. The processing device acquires the natural language input of the structured image to be generated.

[0031] Structured images to be generated refer to professional images that need to be generated and have strict topological and numerical rules, including but not limited to chemical molecular structure diagrams, statistical charts, flowcharts, scientific diagrams, etc.

[0032] Natural language input refers to descriptive text entered by users in everyday language to describe the content, structure, constraints, and style requirements of structured images. For example, it can be used to draw a molecular structure diagram of a benzene ring linked to a nitro group, generate a temperature-yield scatter plot, and display the fitted straight line.

[0033] The processing device actively receives and loads natural language description text input by the user, which explicitly specifies the content, structural relationships, constraints, and presentation requirements of the structured image to be generated, through data input interfaces, human-computer interaction interfaces, or program calls. This natural language input is the original basis for the entire generation process. After the processing device completes the acquisition operation, it will temporarily store and preprocess the text, providing a reliable input foundation for subsequent steps such as semantic parsing, constraint extraction, and symbol generation, ensuring that all subsequent steps revolve around the user's actual needs.

[0034] S202. The processing device performs semantic parsing based on the natural language input to obtain a set of formal specifications and constraints.

[0035] Semantic parsing refers to the process of transforming natural language text into a formal representation that is understandable and executable by machines. This process includes a series of natural language processing operations such as word segmentation, syntactic analysis, semantic role labeling, entity recognition, and relation extraction to extract structured information elements. In this application, the task of semantic parsing is to identify the task domain (such as the chemical molecular domain or the statistical chart domain), structural objects (such as atoms, chemical bonds, coordinate axes, and data points), relationships between objects (such as connection relationships and mapping relationships), and various constraints (such as valence bond rules and coordinate ranges) from the unstructured natural language input by the user, providing clear guidance for subsequent symbol generation and constraint verification.

[0036] Formal specifications refer to standardized requirements documents or data structures output by a semantic parser that guide subsequent symbol generation and constraint verification. Formal specifications transform vague and flexible natural language requirements into a machine-readable and executable format. Their content includes: the task domain and output carrier type (e.g., using SMILES carriers in the field of chemical molecules), structural and constraint intentions (e.g., benzene rings and nitro groups connected by single bonds), and rendering configurations (e.g., displaying grids and labeling atoms). In this application, formal specifications serve as a bridge between natural language understanding and symbol generation, ensuring that the subsequent generation process strictly adheres to the user's true intent.

[0037] A constraint set refers to a set of computable decision rules compiled by a semantic parser based on hard requirements and domain rules in user input. Each constraint in the constraint set can be a predicate function used to determine whether the intermediate symbol representation meets specific topological, syntactic, or numerical requirements. Constraint sets are divided into two categories: hard constraints and soft constraints. Hard constraints are conditions that must be met, such as valence bond rules in chemical molecules or the consistency of the mapping between data points and coordinate axes in a graph. Violating hard constraints will lead to the rejection of the generated result. Soft constraints are conditions that are preferred to be met, such as whether to display a grid or whether to label a legend. Violating soft constraints only affects the energy score and will not lead to direct rejection. In this application, the constraint set is the basis for the symbol validator to perform legality verification.

[0038] Specifically, first, the processing device determines the carrier type and task domain represented by the intermediate symbols based on the natural language input.

[0039] A task domain refers to the category of a structured image described by the user's natural language input, such as a chemical molecular structure diagram, a statistical chart, or a flowchart. Different task domains correspond to different structural rules and constraint systems.

[0040] The carrier type of intermediate symbols refers to the symbolic representation format used to express structured image content, such as the SMILES string or molecular diagram structure in a chemical molecule scene, or Matplotlib code or chart object graph in a statistical chart scene. The carrier type determines the specific methods of subsequent generation, verification, and rendering.

[0041] Let variable d represent the task domain, and the range of values ​​for d is:

[0042] in, In the field of representing chemical molecular structure diagrams, Indicating the field of statistical charts, It can represent other structured image domains, such as flowcharts and circuit diagrams.

[0043] Let variable r represent the carrier type indicated by the intermediate symbol, and the range of values ​​for r is:

[0044] in, It is a linear string used to represent the structure of chemical molecules. A molecular diagram represents the molecular structure, where nodes represent atoms and edges represent chemical bonds. This refers to Python's plotting library code, used to generate statistical charts. It represents the graph structure of chart objects, modeling chart elements as a graph structure of nodes and edges.

[0045] use Let X represent the posterior probability of a task domain of d and a carrier type of r represented by an intermediate symbol, given a natural language input X.

[0046] The combination of task domain and carrier type that maximizes the posterior probability is selected as the judgment result, expressed by the formula:

[0047] in, This represents the optimal combination of task domain and carrier type, i.e., the final judgment result.

[0048] When a user inputs the task of drawing a molecular structure of a benzene ring linked to a nitro group, the processing device identifies the task domain as the chemical molecular domain through keyword recognition (such as benzene ring, nitro group, molecular structure), and accordingly selects SMILES or a molecular diagram as the carrier type for intermediate symbol representation. When a user inputs the task of drawing a scatter plot with temperature on the horizontal axis and yield on the vertical axis, the processing device identifies the task domain as the statistical chart domain and selects Matplotlib code or a chart object as the carrier type. This determination directly affects the instantiation method of the subsequent constraint set and the selection of the renderer, ensuring that the subsequent processing flow matches the user's requirements.

[0049] Then, the processing device extracts the object set, relation set, and attribute set according to the carrier type and task domain represented by the intermediate symbols; based on the object set, relation set, and attribute set, it generates a formal specification, which includes the structure and constraint intent.

[0050] An object set refers to the types and quantities of entities extracted from natural language input that need to appear in the final structured image. Examples include benzene rings, nitro groups, and carbon atoms in a chemical molecule, and the horizontal axis, vertical axis, scatter plots, and fitted lines in a statistical chart. The expression for an object set is:

[0051] in, Represents a collection of objects. For the first The type of an object, For the number of objects, This is a fragment of textual evidence. This indicates the total number of objects in the collection.

[0052] A relation set refers to the structural connections or logical relationships that must be satisfied between objects extracted from natural language input. For example, a benzene ring connected to a nitro group in a chemical molecule, or scatter points in a statistical chart belonging to a data sequence and mapped to coordinate axes. The expression for a relation set is:

[0053] in, Represents a set of relations. and They represent the first in the collection of objects. The and the first The type of an object, Indicates the relation type, Represents relational parameters.

[0054] An attribute set refers to the numerical values, units, styles, and other parameters of an object extracted from natural language input. These can be labeled as hard or soft constraints. Examples include the expression of aromaticity and single bond connections in a chemical molecule, and temperature and grid display on the horizontal axis of a statistical chart. The expression for an attribute set is:

[0055] in, Represents a collection of attributes. Indicates the attribute name. The value represents the attribute value. "hard" or "soft" indicates the attribute type, that is, whether the attribute belongs to a hard constraint or a soft constraint. "hard" indicates a hard constraint (must be satisfied), and "soft" indicates a soft constraint (preferably satisfied).

[0056] Structure and constraint intents refer to the components of a formal specification used to describe the structural objects that should be included in the target image, the relationships between objects, and the constraints extracted from user requirements. These will subsequently be compiled into an executable set of constraints.

[0057] The processing device first determines the extraction strategy and focus based on the identified carrier type and task domain. For example, if the carrier type is SMILES and the task domain is the field of chemical molecules, the extraction will focus on objects such as atoms, chemical bonds, and ring structures and their relationships; if the carrier type is Matplotlib code and the task domain is the field of statistical charts, the extraction will focus on objects such as coordinate axes, data points, and legends and their mapping relationships.

[0058] Then, the processing device extracts three types of information from the natural language text: a set of objects, i.e., the types and quantities of entities that need to appear; a set of relationships, i.e., the structural connections or logical relationships that objects must satisfy; and a set of attributes, i.e., the numerical values, units, styles, and other parameters of the objects, and labels whether the attribute belongs to a hard constraint or a soft constraint.

[0059] Finally, the processing device integrates the extracted object set, relation set, and attribute set to generate the structure and constraint intent section of the formal specification. This section clearly describes which structural objects should be included in the target image, what relationships should exist between these objects, and the user-mandated constraints. This structure and constraint intent will then be compiled into an executable constraint set to guide symbol generation and constraint verification.

[0060] The formal specification expression is:

[0061] in, Formal specifications, This indicates the task domain and output carrier selection, specifying the task domain determined from the natural language input (e.g., chemical molecules, statistical charts) and the type of carrier used for intermediate symbol representation. It represents the structure and constraints, including the set of objects, the set of relations, the set of attributes, and the constraints. This indicates the rendering configuration, including rendering preferences such as displaying the grid, annotation legends, and color settings.

[0062] use The mapping function of the semantic parser maps the natural language input X to a formal specification. :

[0063] For example, given a natural language input of drawing a molecular structure of a benzene ring connected to a nitro group, the generated formal specification's structural and constraint intent can be expressed as follows: it must contain a benzene ring substructure, it must contain a nitro group substructure, there must be a connecting edge between the benzene ring and the nitro group, and the valence bond rules must be satisfied. This formal specification will then be compiled into an executable set of constraints to guide symbol generation and constraint verification.

[0064] Finally, the processing device compiles the structure and constraint intent into a constraint set, which includes both hard and soft constraints.

[0065] Compilation refers to the process of transforming the structure and constraints of a formal specification into a set of rules that can be executed and computed by a machine. This process is similar to converting a high-level language into low-level instructions, enabling the constraints to be directly invoked and evaluated by a symbolic verifier.

[0066] The expression for the constraint set is:

[0067] in, Represents a set of constraints. This represents the i-th constraint predicate function. This represents the instantiation parameter of the i-th constraint. This represents the penalty weight for the i-th constraint. `hard / soft` indicates the attribute type, i.e., whether the attribute belongs to a hard or soft constraint. This indicates the total number of constraints in the constraint set.

[0068] The constraint predicate function is defined as follows: Given a graph structure as input, output one of two results: For hard constraints, output 0 to indicate constraint satisfaction and output 1 to indicate constraint violation; for soft constraints, output a non-negative real number, where 0 indicates complete satisfaction and a larger value indicates a more severe violation. The expression for the constraint predicate function is:

[0069] in, Representing the graph structure, This represents the output space of hard constraints, where 0 indicates that the graph structure satisfies the constraints. 1 indicates that the graph structure violates constraints. It represents the set of nonnegative real numbers.

[0070] Hard constraints are constraints that must be met unconditionally. Violating hard constraints will result in the rejection of the generated result and trigger a regeneration. Hard constraints typically correspond to fundamental rules in a domain or requirements explicitly emphasized by the user, such as valence bond rules in chemical molecules, the consistency of the mapping between data points and coordinate axes in a graph, and the correctness of the parsing syntax.

[0071] Soft constraints are constraints that are preferred but not mandatory to satisfy. Violating soft constraints does not directly reject the generated result, but rather increases the constraint violation energy value, which is used to select the optimal result from multiple candidates. Soft constraints often correspond to rendering preferences or style requirements, such as whether to display a mesh, whether to label legends, and whether to highlight aromatic rings.

[0072] After generating a formal specification containing structural and constraint intentions, the processing device further compiles the constraints described therein into decision rules that can be directly executed by the symbolic verifier. For example, for the constraint intention of a benzene ring connected to a nitro group, the compilation results in a substructure existence constraint, requiring that a benzene ring and a nitro group must exist simultaneously in the molecular diagram and that there must be a connection path between them; for the constraint intention that the valence bond rule must satisfy, the compilation results in a valence bond constraint, requiring that the number of connections for each atom does not exceed its allowed number of valence bonds; for the constraint intention that the coordinates of data points in a statistical chart must be consistent with the coordinate axis scale, the compilation results in a coordinate mapping consistency constraint, requiring that the geometric position of each data point is equal to the result of its data value transformed by the coordinate mapping function.

[0073] The expression for the valence bond constraint is:

[0074] in, This represents the output of the valence key constraint predicate function. Represents a set of atomic nodes. Represents a node The degree, Indicates the type of atom. Indicates atom type Maximum number of valence bonds allowed Indicates an indicator function.

[0075] The expression for the coordinate mapping consistency constraint is:

[0076] in, This represents the output of the coordinate mapping constraint predicate function. Represents a set of data point nodes. Represents a node The geometric position of 2 Represents the coordinate mapping function. Represents a node Data values, Indicates the coordinate axis parameters.

[0077] During compilation, the processing device classifies constraints into hard or soft constraints based on the way the user inputs and the attributes of the constraints themselves. If the user uses mandatory wording such as "must," "required," or "required," or if the constraint involves basic rules of the domain such as valence key rules or coordinate mapping consistency, it is marked as a hard constraint. If the user uses suggestive wording such as "best," "can," "display," or "annotate," or if the constraint involves rendering styles such as legends, grids, or colors, it is marked as a soft constraint.

[0078] After compilation, hard constraints drive rejection sampling and iterative correction, ensuring the generated results meet basic legality requirements; soft constraints guide selection, choosing the output that best matches the user's preferences from multiple candidate results. Through this compilation process, the user's fuzzy natural language constraints are transformed into precise, machine-executable decision rules.

[0079] S203. The processing equipment generates intermediate symbol representations based on formal specifications and constraint sets using a constraint-guided approach.

[0080] Constraint-guided methods refer to intervening in and restricting the generation process of intermediate symbol representations using hard constraints from the constraint set. This ensures that the generation results are directed towards satisfying the constraints from the outset, avoiding the generation of obviously invalid candidates. Specific mechanisms include token-level feasibility filtering, probability distribution renormalization, and constraint beam search.

[0081] Intermediate symbolic representation refers to the structured symbolic product generated from natural language input, serving as a bridge between natural language understanding and deterministic rendering. Intermediate symbolic representation can be expressed using various carrier types, such as the SMILES string or molecular graph structure in a chemical molecule scenario, or Matplotlib code or graph object diagrams in a statistical chart scenario.

[0082] Specifically, firstly, the processing device establishes an autoregressive probability distribution of the symbol sequence according to formal specifications.

[0083] A symbol sequence is a sequence of symbols that represents intermediate symbols. For example, the string SMILES can be viewed as a sequence of character symbols, and Matplotlib code can be viewed as a sequence of code tokens.

[0084] Autoregressive probability distribution is a probabilistic modeling method for sequence generation. The idea is that the generation probability of each symbol in the sequence depends on all the symbols that have been generated before. In other words, the generation process is carried out symbol by symbol, and the generation condition of the current symbol depends on the historical context.

[0085] The processing device determines the specific format of intermediate symbol representation and the vocabulary based on the task domain and carrier type in the formal specification. For example, if the task domain is determined to be in the field of chemical molecules and the carrier type is SMILES, the vocabulary includes symbols such as carbon, nitrogen, oxygen, single bonds, double bonds, parentheses, and ring labels; if the task domain is determined to be in the field of statistical charts and graphs and the carrier type is Matplotlib code, the vocabulary includes code tokens such as import statements, plotting functions, and axis labels.

[0086] The processing device then establishes an autoregressive probability distribution, which decomposes the joint probability of the entire symbol sequence into a product of conditional probabilities at each step. This probability distribution is parameterized by a generator model (such as a large language model), whose inputs include formal specifications and previously generated historical symbols, and whose output is the probability distribution of the next symbol.

[0087] The expression for the autoregressive probability distribution is:

[0088]

[0089] in, This indicates that, given a formal specification Under the condition that the entire symbol sequence The autoregressive probability distribution, This represents the parameters of the generator model (such as the weights of a neural network). This indicates that, given a formal specification and generated prefix Under the conditions, the first Symbols at each position The conditional probability, The symbol representing the t-th position. This indicates that a prefix has been generated, that is, the sequence of historical symbols generated up to the t-th position. , This represents the total length of the entire symbol sequence S; This represents the symbol at position 1 in the entire symbol sequence. This represents the symbol at the second position in the entire symbol sequence. This represents the symbol at the last position in the entire symbol sequence. This represents the symbol at position t-1 in the entire symbol sequence.

[0090] The corresponding log-likelihood is:

[0091] in, This indicates that, given a formal specification Under the condition that the entire symbol sequence The log-likelihood value, This indicates that, given a formal specification and generated prefix Under the conditions, the first Symbols at each position The logarithm of the conditional probability, This represents the total length of the entire symbol sequence S.

[0092] Log-likelihood transforms the multiplication operation in the autoregressive probability distribution described above into a summation operation. Since the original probability is the product of multiple conditional probabilities, direct calculation can easily lead to numerical underflow (i.e., the product is too small for the computer to accurately represent). Taking the logarithm transforms the product into a summation, avoiding underflow and preserving the monotonicity of the probability (i.e., the higher the probability, the higher the log-likelihood). This log-likelihood function is often used in the training process of generator models as an optimization objective to maximize the probability of generating correct sequences.

[0093] Then, the processing device selects a feasible set of symbols from the vocabulary based on the hard constraints in the constraint set.

[0094] A vocabulary is the set of all possible symbols and serves as the output space of a generator model during autoregressive decoding. Each symbol in the vocabulary represents a basic generation unit. For example, in SMILES generation, the vocabulary includes characters such as carbon, nitrogen, oxygen, single bonds, double bonds, parentheses, and ring symbols; in Matplotlib code generation, the vocabulary includes code tokens such as import statements, function names, parameter names, and operators.

[0095] The feasible symbol set refers to the set of symbols selected from the vocabulary that do not violate any hard constraints in the current decoding step. The feasible symbol set is a subset of the vocabulary, where each symbol satisfies all hard constraints under the currently generated prefix conditions.

[0096] In each step of autoregressive decoding, the processing device first obtains the vocabulary of the current step as a candidate symbol set. Then, based on the hard constraints in the constraint set, the processing device performs layer-by-layer filtering of the candidate symbol set from three dimensions, and finally obtains the intersection of the three sets as the feasible symbol set.

[0097] The first dimension is the set of grammatically valid symbols, which ensures that the sequence of symbols conforms to basic grammatical rules, such as parentheses must be closed, quotation marks must be paired, and ring marks in SMILES must appear in pairs. Only symbols that satisfy these grammatical rules can be included in this set.

[0098] The second dimension is the set of format constraints, determined by the task domain and carrier type. These constraints ensure that symbol sequences conform to the basic format requirements of a specific domain. For example, chemical molecule generation must produce resolvable SMILES, and statistical chart code generation must include necessary import statements and plotting function calls. Only symbols that meet these format constraints can be included in this set.

[0099] The third dimension is the forward constraint pruning set, used to preemptively eliminate symbols that seem legal at the current step but will inevitably lead to future constraint violations. For example, in SMILES generation, if the atoms already generated are close to their valence bond limit, then selecting any symbol that introduces additional chemical bonds will cause valence bond exceedances in the future, and such symbols will be preemptively removed by forward pruning.

[0100] The processing device evaluates each symbol in the candidate symbol set according to the three dimensions mentioned above. Only symbols that simultaneously belong to the set of valid syntax, the set of format constraints, and the set of forward-looking constraint pruning are retained. After these three layers of screening, the processing device obtains the feasible symbol set, which is the intersection of the three sets.

[0101] This feasible symbol set forms the basis for subsequent renormalization operations, ensuring that the generation process is carried out only within the valid symbol space. This eliminates illegal branches at the token level in advance, improving generation efficiency and first-pass rate.

[0102] The definition expression for the feasible symbol set is:

[0103] in, Indicates that in a given and The following feasible symbol set, This represents the set of valid syntax formats. This represents the set of format constraints determined by the task domain and the carrier type. Represents the set of forward-looking constrained pruning. The symbol represents the intersection.

[0104] The processing device renormalizes the autoregressive probability distribution based on the feasible symbol set.

[0105] At position t of the autoregressive decoding, the generator model first outputs a raw probability distribution covering the entire vocabulary based on the formal specifications and the generated historical symbols. In this distribution, each symbol is assigned a probability value, and the sum of the probabilities of all symbols is 1. However, this raw distribution may contain some illegal symbols that violate hard constraints in the current context.

[0106] The processing device renormalizes the original probability distribution based on the feasible symbol set obtained in the previous step. First, it determines whether the current symbol belongs to the feasible symbol set using an indicator function: if it does, the indicator function is set to 1, retaining its original probability; if it does not belong, the indicator function is set to 0, setting its probability to zero, thus prohibiting the selection of these illegal symbols. Then, the probabilities of the symbols in the feasible symbol set are rescaled until their sum equals 1. The renormalization formula is: the new probability of each symbol in the feasible symbol set equals its original probability multiplied by the indicator function, divided by the sum of the original probabilities of all symbols in the feasible symbol set. The denominator serves as a normalization factor, ensuring that the sum of the probabilities of all feasible symbols after renormalization is 1.

[0107] After renormalization, the processing device obtains a corrected probability distribution that covers only the feasible symbol set. In this distribution, the probability of an illegal symbol is zero, and the sum of the probabilities of legal symbols is 1. The processing device then samples according to this corrected probability distribution and selects the symbol at position t. Through the renormalization operation, the processing device reduces the probability of illegal branches to zero at the token level, ensuring that the generation process only occurs within the legal space that satisfies the hard constraints, thereby eliminating a large number of obviously illegal candidates during the generation stage.

[0108] The expression for renormalization is:

[0109] in, This represents the probability distribution after renormalization. This indicates that, given a formal specification and generated prefix Under the conditions, the first Symbols at each position The conditional probability, Indicates that in a given and The following feasible symbol set, This indicates that, given a formal specification and generated prefix Under the condition, symbol The conditional probability is calculated by summing the original probabilities of all candidate symbols in the feasible symbol set in the denominator.

[0110] Equivalent land, if used The model represents the first time. The unnormalized output (logits) at each position can be rewritten as:

[0111] in, Indicates the first Symbols at each position The corrected probability, i.e. the probability distribution after renormalization. The model represents the first Symbols at each position The original logit value, The model represents the first time. Each position pair of symbols The original logit value, Iterate through each symbol in the set of feasible symbols. Represents the feasible symbol set.

[0112] Finally, the processing device decodes the intermediate symbol representation based on the renormalized probability distribution.

[0113] Specifically, at the t-th decoding position, the processing device has completed the following operations: establishing an autoregressive probability distribution, filtering feasible symbol sets according to hard constraints, and renormalizing the probability distribution. At this point, the processing device obtains a corrected probability distribution that only covers the feasible symbol set and whose probability sum is 1.

[0114] Based on the renormalized probability distribution, the processing device selects the symbol at position t using a preset decoding strategy. Common decoding strategies include: random sampling based on probability (i.e., symbols with higher probabilities are more likely to be selected); selecting the symbol with the highest probability (i.e., greedy decoding); or retaining multiple candidate prefixes (i.e., beam search). After selection, the processing device adds the selected symbol to the end of the generated prefix, obtaining a new generated prefix, which is used for generating the next decoding position (i.e., position t+1).

[0115] The processing device repeats the above process, that is, it performs feasibility filtering, renormalization, and symbol selection at each decoding position until a complete symbol sequence is generated. When a sequence terminator is encountered or the preset maximum length is reached, the decoding process terminates, and the resulting complete symbol sequence is the intermediate symbol representation.

[0116] By using this constraint-guided decoding method, the processing device ensures that the generated intermediate symbol representation meets the hard constraint requirements at every decoding position, thereby eliminating a large number of obviously invalid candidates during the generation stage, improving generation efficiency and first-pass yield, and laying a good foundation for subsequent symbol verifier verification.

[0117] Constraints in structured symbolic representations often exhibit the characteristic of locking in subsequent feasibility through early decisions. Taking SMILES as an example, loop closure labels must appear in pairs; branch brackets must be closed; and certain atomic connection methods, once selected, can lead to valence bond overruns later. In Matplotlib code, for instance, if a chart type inconsistent with the formal specifications is selected (e.g., the main statement generates a line chart when a scatter plot is required), or a plotting function that doesn't match the data dimensions is chosen, even subsequent remediation will struggle to meet numerical consistency and executability requirements. If these problems are only discovered by the validator after generation, it will result in a large number of invalid candidates.

[0118] Therefore, this method introduces a lightweight rule filter or predictor at the t-th decoding position to evaluate the future feasibility of each candidate symbol under the current generated history and formal specifications. A local feasibility decision function is defined, with an output of 0 or 1: 0 indicates that the candidate symbol will cause the sequence to enter an irreparable or highly probable irreparable state, and 1 indicates that it is feasible. Based on this decision result, candidate symbols with an output of 0 are removed from the set of feasible symbols.

[0119] To avoid excessively expensive computation of the local feasibility decision function, this application restricts look-ahead pruning to constraints that are locally decisionable or can be predicted by a finite number of steps, typically including the following: Pairing and Stack Constraints (General): For parentheses, quotation marks, SMILES branch parentheses, and ring labels, a stack state can be maintained. If choosing a candidate symbol would cause the stack state to become unreachable (e.g., redundant closing parentheses, duplicate illegal ring labels), the decision function outputs 0. This type of decision can be formalized as a reachability update of a deterministic finite automaton or a pushdown automaton.

[0120] Local semantic consistency constraints (domain-dependent): Key slots in the formal specification output by the semantic parser should be progressively covered during generation. A coverage function is defined to measure the proportion of required slots covered by generated prefixes. Pruning is possible if a candidate symbol would prevent subsequent coverage of required slots within the remaining length budget (e.g., most symbols have been consumed but no scatter data source or axis definition has been introduced). This decision is made by comparing the remaining length budget with the minimum generation cost.

[0121] Repairability constraints (for closed-loop correction efficiency): Some errors can theoretically be remedied later (e.g., missing headings can be added), while others are almost irreparable once they occur (e.g., SMILES ring label conflicts, broken Python code indentation structure). This application focuses pruning on unrecoverable errors to avoid excessive constraints that could reduce the generator's expressive power. Repairability is determined by classifying error types into "fatal errors" and "recoverable errors" using a rule base, and the decision function outputs 0 for fatal errors.

[0122] Example from a chemical molecule scenario: The direct benefit of look-ahead pruning is a significant reduction in the proportion of candidates that are syntactically parsable but topologically incompatible. For instance, when the current partial molecular graph has brought an atom close to its valence limit, if the next candidate symbol would introduce an additional bond type leading to overvalence, it can be pruned directly at the symbol level. This allows for maintaining a defined lower bound on the atomic degree through the local molecular graph state and evaluating whether candidate symbols necessarily lead to a violation. The lower bound on the degree refers to the minimum degree that must be formed under the current partial parsing, enabling pruning to work without requiring a complete molecular parsing.

[0123] Example of a statistical chart scenario: Lookahead pruning primarily avoids generating non-executable or contradictory code paths. For instance, when the formal specification specifies a scatter plot type and requires line fitting, if the generator selects a contradictory function family when generating the main plotting statement (e.g., generating only a bar chart), it will be pruned. Similarly, when the formal specification specifies the number of data points or the data source format, choices that lead to inconsistent dimensions (e.g., significantly unequal lengths of the x and y axes) can also be pruned beforehand. This type of pruning does not require actual code execution but can be achieved through localized determination using static abstract syntax tree pattern matching.

[0124] Symbolic pruning alone can eliminate obviously illegal branches, but it is still insufficient to guarantee that global constraints (especially topological and numerical consistency) will eventually be satisfied. To increase the probability of passing verification in one generation, this application employs constrained beam search to retain multiple candidate prefixes and rank them by score during the generation stage, making the search tend to favor candidates that simultaneously satisfy both high language model probability and low constraint energy.

[0125] The beamwidth determines the number of candidate prefixes retained at each step. For any prefix, its generation score is defined as the cumulative log probability.

[0126] A partial constraint cost is defined to estimate the risk of future constraint violations in the prefix. Since the energy of global constraint violations cannot be fully calculated before the sequence is complete, this application constructs an upper bound or surrogate cost. If the prefix has already violated an irreparable hard constraint, the partial constraint cost is set to positive infinity; if it has not yet been violated, a non-negative cost is given based on look-ahead pruning and the local structural state. This surrogate cost consists of two parts: one is a weighted sum of definite constraint violations on the prefix, and the other is a risk term based on uncertainties estimated by missing slots, unclosed structures, or static analysis, multiplied by a tradeoff coefficient.

[0127] Therefore, constrained beam search ranks candidates based on a joint objective. This joint objective equals the language model score minus the constraint strength coefficient multiplied by a portion of the constraint cost. The constraint strength coefficient controls the weight of the constraint cost in the joint score and is non-negative.

[0128] At each expansion step, only symbols in the feasible symbol set are expanded, and the prefixes with the highest joint scores are selected from all expansion candidates to proceed to the next step. This results in several complete sequence candidates, which are then fed into the symbol verifier to calculate the actual constraint violation energy, and a final selection is made or feedback corrections are initiated.

[0129] The advantage of this layered approach of generation and validation phases lies in the fact that the generation phase uses inexpensive proxy costs to quickly shrink the search space, while the validation phase uses strict external rules for final judgment. This combination ensures correctness while controlling computational costs. Especially in statistical chart scenarios, static proxy costs can significantly reduce runtime errors and candidates with obvious dimensional mismatches; in chemical molecule scenarios, proxy costs can suppress candidates with obvious overvalence and disordered ring labels, thereby improving the first-pass yield.

[0130] S204. The processing device obtains the constraint violation energy and error information through the symbol verifier based on the intermediate symbol representation and the constraint set.

[0131] The symbolic verifier is a non-differentiable, executable rule engine or solver engine used to rigorously verify the constraints of intermediate symbolic representations. The symbolic verifier takes intermediate symbolic representations, formal specifications, and constraint sets as input, and outputs constraint violation energy, error set, and evidence chain. The symbolic verifier is the component in this application that enables provable structural correctness.

[0132] Constraint violation energy is a quantitative indicator obtained by weighted summation of the degree of violation of each constraint in the constraint set represented by intermediate symbols. The expression for constraint violation energy is the weighted sum of the degree of violation of each constraint and its corresponding penalty weight, used to measure the overall legality of the intermediate symbol representation.

[0133] Error messages are detailed descriptions of violated constraints output by the symbol validator. They include the error type, the location of the error in the intermediate symbol representation, the error description, and suggested corrections, which drive subsequent feedback loop adjustments.

[0134] Specifically, the processing device parses the intermediate symbol representation and maps the parsing result to a verifiable structural kernel; based on the constraint set, the structural kernel is constrained and verified to obtain the constraint violation energy and error information respectively.

[0135] After generating the intermediate symbolic representation, the processing device inputs this intermediate symbolic representation along with the constraint set obtained in the semantic parsing stage into the symbol verifier. The symbol verifier first parses the intermediate symbolic representation, mapping it to a verifiable structural kernel. For example, for the string "SMILES" in a chemical molecule scenario, the symbol verifier parses it into a molecular graph structure; for Matplotlib code in a statistical chart scenario, the symbol verifier parses it into an abstract syntax tree and extracts the chart object structural kernel.

[0136] Then, the symbolic validator verifies the structural kernel one by one for each constraint in the constraint set. For hard constraints, the symbolic validator determines whether a constraint is violated; if violated, it is marked as 1, otherwise as 0. For soft constraints, the symbolic validator calculates the degree of violation and outputs a non-negative real number. The symbolic validator then performs a weighted summation of the violation degrees of all constraints with their corresponding penalty weights to obtain the constraint violation energy. Simultaneously, for each violated constraint, the symbolic validator generates structured error information, including the error type, the location of the error in the intermediate symbolic representation, the error description, and suggested modifications.

[0137] Through the verification by the symbol validator, the processing device obtains two key outputs: constraint violation energy, used to quantify the overall legality of the intermediate symbol representation; and error information, used to locate the specific error location and cause. These two outputs will be used in subsequent feedback loop correction steps. When the constraint violation energy exceeds a threshold, the processing device generates error correction prompts based on the error information and regenerates the intermediate symbol representation until a legal result is obtained.

[0138] The expression for constraint violation energy is:

[0139] in, This indicates that the constraint violates energy. Represents a set of constraints. Indicates the first The penalty weight of each constraint, Indicates the first The degree of violation of a constraint.

[0140] The expression for the error set is:

[0141] in, Represents the set of errors. Indicates the first An error message, Indicates the error type. Indicates incorrect location. This indicates an incorrect description. Indicates suggestions for modification. This indicates the total number of error messages.

[0142] S205. The processing device obtains a valid intermediate symbol representation based on the constraint violation energy and error information.

[0143] A valid intermediate symbol representation is an intermediate symbol representation that has been verified by a symbol validator and satisfies all hard constraints (i.e., constraint violation energy is zero or less than or equal to the energy threshold). Valid intermediate symbol representations serve as input for subsequent deterministic rendering, ensuring that the final output structured image strictly satisfies domain rules in terms of topological structure and numerical relationships.

[0144] Specifically, when the constraint violation energy is less than or equal to the energy threshold, the current intermediate symbol representation is taken as a valid intermediate symbol representation; when the constraint violation energy is greater than the energy threshold, if the current iteration round has not reached the maximum round, an error correction prompt is generated based on the error information, and the intermediate symbol representation is regenerated based on the error correction prompt and formal specifications; if the current iteration round has reached the maximum round, the intermediate symbol representation with the lowest constraint violation energy is selected from the multiple generated intermediate symbol representations as a valid intermediate symbol representation.

[0145] After obtaining the verification result from the symbol validator, the processing device first determines whether the constraint violation energy is less than or equal to the energy threshold. If the constraint violation energy is less than or equal to the energy threshold, it means that the current intermediate symbol representation has satisfied all hard constraints, and the processing device directly uses it as a valid intermediate symbol representation to proceed to the subsequent deterministic rendering steps.

[0146] If the constraint violation energy exceeds the energy threshold, it indicates that the current intermediate symbol represents a hard constraint violation or an excessive deviation from the soft constraint. At this point, the processing device further determines whether the current iteration has reached the preset maximum iteration. If the maximum iteration has not been reached, the processing device generates an error correction prompt based on the error information. The formula for generating the error correction prompt is:

[0147] in, Indicates the first The error correction hints generated in each iteration are used to guide the generator in correcting the intermediate symbol representation in the next iteration. This function represents a formatted function that converts error messages and chains of evidence into error correction suggestions. Indicates the first The error set of the round of iteration Indicates the first Evidence chain or meta-information of round iteration, Represents formal specifications.

[0148] Then, the processing device uses the error correction hints and formal specifications together as conditions to drive the generator to regenerate the intermediate symbol representation. The regenerated formula is:

[0149] in, Indicates the first Intermediate symbols generated by round iterations are represented as follows: Represented by formal specifications and error correction tips The probability distribution of generation is conditional.

[0150] Then, the processing device performs symbol verification again, forming an iterative closed loop of generation, verification, feedback, and correction.

[0151] If the current iteration round has reached the preset maximum number of rounds, to avoid infinite loops, the processing device stops iterating and selects the intermediate symbol representation with the lowest constraint violation energy from all generated intermediate symbol representations as the legal intermediate symbol representation. The selection formula is as follows:

[0152] in, This represents a valid intermediate symbol. Indicates the first The constraint violation energy is represented by a candidate intermediate symbol. This means selecting the candidate that minimizes the energy value from all candidates.

[0153] When multiple candidate constraints have equal energy of violation, the language model score can be introduced as a secondary ranking term to select the result with the higher probability of generation.

[0154] in, Represents the balance coefficient. This represents the log-likelihood of the generative model with respect to the symbol sequence.

[0155] The result obtained at this point may not fully satisfy all soft constraints, but it will at least satisfy all hard constraints, or it may be the optimal result within a finite number of rounds.

[0156] Through this feedback loop mechanism, the processing device can automatically locate the cause of the error and continuously optimize within a limited number of rounds until a valid intermediate symbol is obtained, which not only ensures the validity of the final output, but also avoids the waste of computing resources caused by infinite retries.

[0157] The energy threshold is obtained in the following way: The processing device determines the energy threshold based on the current iteration round, the historical best energy value, and the convergence parameters.

[0158] The energy threshold refers to the upper limit of constraint violation energy used to determine whether an intermediate symbol representation is acceptable. When the constraint violation energy is less than or equal to the energy threshold, the current intermediate symbol representation is considered a valid result and proceeds directly to the rendering step; when the constraint violation energy is greater than the energy threshold, a feedback correction process is triggered.

[0159] The historical optimal energy value refers to the minimum constraint violation energy among all generated intermediate symbol representations from the first iteration to the current iteration. The historical optimal energy value reflects the best legality the system can achieve so far and is an important reference for dynamically adjusting the energy threshold.

[0160] Convergence parameters are configurable parameters used to control the rate and magnitude of change of the energy threshold with each iteration. Convergence parameters can include initial threshold, decay factor, residual coefficient, etc., and are used to balance generation quality and convergence efficiency.

[0161] Specifically, in each iteration of the feedback loop, the processing device first obtains three inputs: the current iteration number, indicating how many correction attempts the system has made; the historical best energy value, indicating the best result generated so far; and a preset convergence parameter, used to control the change pattern of the threshold.

[0162] The processing device then calculates the energy threshold for the current round based on these inputs. In the early stages of iteration, the energy threshold can be set relatively loosely, allowing for a certain degree of soft constraint deviation to quickly obtain feasible results. As the number of iterations increases, the energy threshold gradually tightens, requiring the generated results to be increasingly closer to complete validity. Historical optimal energy values ​​are also incorporated into the calculation; when the historical optimal energy value is already small, the energy threshold can be lowered accordingly to avoid performing too many meaningless iterations when the result is already close to optimal.

[0163] Through this dynamic threshold adjustment mechanism, the processing device can achieve a balance between generation quality and convergence efficiency: ensuring the validity of the final output while avoiding infinite retries due to overly strict threshold settings, or accepting poor-quality results due to overly lenient threshold settings.

[0164] The formula for calculating the energy threshold is:

[0165] in, Indicates the first Energy threshold for round iteration Indicates the initial threshold. Represents the attenuation factor, and , Denotes the residual coefficient, and , This represents the historical best energy value, and the convergence parameters include the initial threshold, decay factor, and residual coefficient.

[0166] S206. The processing device obtains a structured image through deterministic rendering mapping based on the legal intermediate symbols.

[0167] Deterministic rendering mapping refers to the process of converting a valid intermediate symbolic representation into a structured image through a deterministic renderer. This mapping is deterministic, meaning that the same valid intermediate symbolic representation will always render the same structured image; there is no randomness or sampling uncertainty.

[0168] Structured images refer to image types with clear topological relationships, numerical mappings, and formally verifiable rules, such as chemical molecular structure diagrams, statistical charts, flowcharts, and circuit diagrams. Structured images are the final visualization deliverables to users.

[0169] Specifically, the processing device invokes a deterministic renderer to render valid intermediate symbol representations into a structured image; and outputs the structured image.

[0170] After obtaining a valid intermediate symbol representation, the processing device first selects an appropriate deterministic renderer based on the task domain and the vector type. If the task domain is in the field of chemical molecules and the vector type is SMILES or a molecular diagram, the processing device calls the chemical molecule rendering engine; if the task domain is in the field of statistical charts and the vector type is Matplotlib code or a chart object, the processing device calls the Python interpreter to execute the plotting code, or outputs the chart object as a raster or vectorized graph.

[0171] Then, the processing device obtains the rendering configuration generated during the semantic parsing phase and passes the valid intermediate symbol representations along with the rendering configuration to the renderer. The rendering configuration includes the user's preferences for image style, such as whether to display a grid, whether to label legends, whether to display atomic labels, color scheme, line type settings, etc., which are used to control the renderer's output style.

[0172] The expression for deterministic rendering mapping is:

[0173] in, This represents the final structured image output. This represents a deterministic renderer function. This represents a valid intermediate symbol. This indicates the rendering configuration.

[0174] The processing device executes the renderer, converting valid intermediate symbolic representations into structured images. Because the renderer is deterministic, the same valid intermediate symbolic representations and rendering configurations always output the same structured image, eliminating randomness in pixel generation and ensuring reproducible results.

[0175] If a syntax error or runtime error occurs during rendering, the processing device will treat the rendering failure as a hard constraint violation and revert the currently valid intermediate symbol representation back to the feedback loop for correction.

[0176] After successful rendering, the processing device outputs the final structured image. The system deliverables also include a valid intermediate symbol representation file, a verification report, and rendering configuration, achieving reproducible, editable, and auditable engineering-grade delivery.

[0177] Based on the above description, this application has the following beneficial effects: In this application, an intermediate symbolic representation is introduced as a bridge between generation and rendering, elevating the generation target from pixel space to symbolic space. At the same time, a symbolic verifier performs strict constraint verification on the intermediate symbolic representation, calculates the constraint violation energy and outputs localizable error information, ensuring that the final output structured image strictly meets the domain rules in terms of topological structure and numerical relationships, fundamentally solving the problem of existing technologies drawing realistically but incorrectly.

[0178] Furthermore, on the one hand, a constraint-guided approach is adopted in the generation phase. Feasible symbol sets are selected from the vocabulary based on hard constraints, and the probability distribution is renormalized, eliminating illegal branches at the token level in advance. On the other hand, in the verification phase, a symbol verifier outputs structured error information. When the constraint violation energy exceeds the energy threshold, error correction prompts are generated based on the error information, driving the generator to regenerate, forming an iterative closed loop of generation, verification, feedback, and correction. This dual verification mechanism enables the system to automatically locate the cause of errors and continuously optimize until a valid intermediate symbol representation is obtained, improving the success rate and stability of generation.

[0179] Finally, this application introduces an energy threshold control mechanism in the feedback loop. When the constraint violation energy is less than or equal to the energy threshold, the current result is directly accepted to avoid unnecessary iterations. When the threshold is exceeded, targeted corrections are made based on the error information. At the same time, the energy threshold is dynamically determined based on the current iteration round, the historical best energy value, and the convergence parameters, so that the system can achieve a balance between generation quality and convergence efficiency, which not only ensures the legality of the final output, but also avoids the waste of computing resources caused by infinite retries.

[0180] The above text combined Figure 1 The structured image generation method provided in the embodiments of this application has been described in detail. The apparatus and equipment provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0181] like Figure 2 As shown in the figure, this is a schematic diagram of a structured image generation apparatus provided in an embodiment of this application. The apparatus includes: The acquisition module 301 is used to acquire the natural language input of the structured image to be generated; Processing module 302 is used to perform semantic parsing based on natural language input to obtain a formal specification and constraint set; generate intermediate symbol representations based on the formal specification and constraint set using a constraint-guided approach; obtain constraint violation energy and error information through a symbol validator based on the intermediate symbol representations and constraint set; and obtain a valid intermediate symbol representation based on the constraint violation energy and error information. The determination module 303 is used to obtain a structured image through deterministic rendering mapping based on the legal intermediate symbol representation.

[0182] Optionally, the processing module 302 is specifically used to treat the current intermediate symbol representation as a valid intermediate symbol representation when the constraint violation energy is less than or equal to the energy threshold. When the energy of the constraint violation is greater than the energy threshold, if the current iteration round has not reached the maximum round, an error correction prompt is generated based on the error information, and an intermediate symbol representation is regenerated based on the error correction prompt and the formal specification. If the current iteration reaches the maximum iteration, select the intermediate symbol representation with the lowest constraint violation energy from the multiple generated intermediate symbol representations as the legal intermediate symbol representation.

[0183] Optionally, the processing module 302 is specifically used to determine the energy threshold based on the current iteration round, the historical best energy value, and the convergence parameters.

[0184] Optionally, the processing module 302 is specifically used to parse the intermediate symbol representation and map the parsing result into a verifiable structure kernel; Based on the set of constraints, the structural kernel is constrained to obtain constraint violation energy and error information.

[0185] Optionally, the processing module 302 is specifically used to determine the carrier type and task domain represented by the intermediate symbols based on the natural language input; Based on the carrier type and task domain represented by the intermediate symbols, extract the object set, relation set, and attribute set; Based on the set of objects, the set of relations, and the set of attributes, a formal specification is generated, which includes the structure and constraint intent. The structure and constraint intent are compiled into a constraint set, which includes hard constraints and soft constraints.

[0186] Optionally, the processing module 302 is specifically used to establish the autoregressive probability distribution of the symbol sequence according to the formal specifications; Based on the hard constraints in the constraint set, a feasible symbol set is selected from the vocabulary; The autoregressive probability distribution is renormalized based on the feasible symbol set; Based on the renormalized probability distribution, the intermediate symbol representation is obtained through decoding.

[0187] Optionally, the determination module 303 is specifically used to call the deterministic renderer to render the legal intermediate symbol representation into a structured image; Output the structured image.

[0188] The structured image generation apparatus according to the embodiments of this application can correspond to the execution of the method described in the embodiments of this application, and the other operations and / or functions of each module / unit of the structured image generation apparatus are respectively for implementing Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiments will not be described in detail here.

[0189] This application also provides a computing device. For example... Figure 3 As shown in the figure, this is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 700 includes a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, the memory 704, and the communication interface 703 communicate with each other via the bus 701.

[0190] The 701 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0191] The processor 702 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0192] The communication interface 703 is used for communication with external devices.

[0193] Memory 704 may include volatile memory, such as random access memory (RAM). Memory 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0194] The memory 704 stores executable code, and the processor 702 executes the executable code to perform the aforementioned structured image generation method.

[0195] Specifically, in achieving Figure 2 In the case of the illustrated embodiment, and Figure 2 When the modules or units of the structured image generation apparatus described in the embodiments are implemented by software, the following steps are performed: Figure 2 The software or program code required for the functions of each module / unit can be partially or wholly stored in memory 704. Processor 702 executes the program code corresponding to each unit stored in memory 704 to execute the aforementioned structured image generation method.

[0196] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to execute the above-described structured image generation method.

[0197] This application also provides a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application are generated.

[0198] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0199] When the computer program product is executed by a computer, the computer performs any of the aforementioned structured image generation methods. The computer program product can be a software installation package; when any of the aforementioned structured image generation methods is required, the computer program product can be downloaded and executed on the computer.

[0200] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0201] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application.

Claims

1. A structured image generation method, characterized in that, The method includes: Obtain the natural language input for the structured image to be generated; Based on the natural language input, semantic parsing is performed to obtain a set of formal specifications and constraints; Based on the formal specifications and constraint set, intermediate symbol representations are generated using a constraint-guided approach. Based on the intermediate symbolic representation and the constraint set, the constraint violation energy and error information are obtained through the symbolic verifier. Based on the constraint violation energy and error information, a valid intermediate symbol representation is obtained; Based on the legal intermediate symbols, a structured image is obtained through deterministic rendering mapping.

2. The method according to claim 1, characterized in that, The process of obtaining a valid intermediate symbol representation based on constraint violation energy and error information includes: When the constraint violates the energy threshold, the current intermediate symbol representation is taken as a valid intermediate symbol representation. When the energy of the constraint violation is greater than the energy threshold, if the current iteration round has not reached the maximum round, an error correction prompt is generated based on the error information, and an intermediate symbol representation is regenerated based on the error correction prompt and the formal specification. If the current iteration reaches the maximum iteration, select the intermediate symbol representation with the lowest constraint violation energy from the multiple generated intermediate symbol representations as the legal intermediate symbol representation.

3. The method according to claim 2, characterized in that, The energy threshold is obtained in the following way: The energy threshold is determined based on the current iteration round, the historical best energy value, and the convergence parameters.

4. The method according to claim 1, characterized in that, The process of obtaining constraint violation energy and error information through a symbolic verifier based on intermediate symbolic representations and constraint sets includes: The intermediate symbol representation is parsed, and the parsing result is mapped to a verifiable structural kernel; Based on the set of constraints, the structural kernel is constrained to obtain constraint violation energy and error information.

5. The method according to claim 1, characterized in that, The process of obtaining a set of formal specifications and constraints through semantic parsing based on natural language input includes: Based on the natural language input, determine the carrier type and task domain represented by the intermediate symbols; Based on the carrier type and task domain represented by the intermediate symbols, extract the object set, relation set, and attribute set; Based on the set of objects, the set of relations, and the set of attributes, a formal specification is generated, which includes the structure and constraint intent. The structure and constraint intent are compiled into a constraint set, which includes hard constraints and soft constraints.

6. The method according to claim 5, characterized in that, The process of generating intermediate symbol representations based on formal specifications and constraint sets using a constraint-guided approach includes: Based on formal specifications, establish the autoregressive probability distribution of the symbol sequence; Based on the hard constraints in the constraint set, a feasible symbol set is selected from the vocabulary; The autoregressive probability distribution is renormalized based on the feasible symbol set; Based on the renormalized probability distribution, the intermediate symbol representation is obtained through decoding.

7. The method according to claim 1, characterized in that, The step of obtaining a structured image through deterministic rendering mapping based on valid intermediate symbols includes: Invoke the deterministic renderer to render the legal intermediate symbol representation into a structured image; Output the structured image.

8. A structured image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire the natural language input for the structured image to be generated; The processing module is used to perform semantic parsing on natural language input to obtain a formal specification and constraint set; generate intermediate symbolic representations using a constraint-guided approach based on the formal specification and constraint set; obtain constraint violation energy and error information through a symbolic verifier based on the intermediate symbolic representations and constraint set; and obtain valid intermediate symbolic representations based on the constraint violation energy and error information. The determination module is used to obtain a structured image through deterministic rendering mapping based on the legal intermediate symbol representation.

9. A computing device, characterized in that, Including memory and processor; The memory stores one or more computer programs, the one or more computer programs including instructions; when the instructions are executed by the processor, the computing device performs the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method as described in any one of claims 1 to 7.

Images