A multi-agent layout generation system based on rule formalization injection

Abstract

The application relates to the field of multi-modal model and multi-agent cooperation, and discloses a multi-agent layout generation system based on rule formalization injection, which comprises the following core steps: a design specification capable of being received by an agent in a natural language input is generated to generate a formal rule library; a VLM multi-modal intention understanding agent performs preliminary analysis after receiving user materials and design requirements, and distributes tasks to downstream professional agents for processing to create an initial layout; subsequently, a formal rule checking agent calls the rule library to perform compliance checking; if the layout is not compliant, an iteration correction cycle is automatically triggered until the layout passes the checking or the iteration upper limit is reached; and the user can manually fine-tune or version rollback on this basis. The subjective design specification is converted into objective machine instructions, and a multi-agent cooperation process is established, so that the specification compliance problem in automatic layout is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of multi-modal model and multi-agent collaboration, and particularly relates to a multi-agent layout generation system based on rule formalization injection. BACKGROUND

[0002] With the rapid development of digital content and information dissemination, the advertising design, publishing production, webpage layout and e-commerce marketing industries have high requirements for the efficiency, quality and standardization of graphic text layout. The traditional layout process, from communicating needs, understanding materials to design implementation and repeated modification, highly depends on manual operation. Due to the natural deviation between the subjectivity of customer needs and the aesthetic understanding of designers, the communication cost is high, the rework rate is high, and the final output is often a compromise result, which is difficult to achieve the best effect.

[0003] To address this challenge, some automatic layout generation systems have appeared on the market. However, these systems generally have two major problems: first, they mostly lack effective human-machine collaboration mechanisms, making it difficult for users to intervene and fine-tune during the generation process; second, and most importantly, layout work in various industries must follow detailed and strict rules (such as brand specifications, publication format standards, advertising law regulations, etc.), and existing automatic systems cannot truly understand these rules, let alone generate compliant layouts under the strict guidance of the rules. Simply inputting rules as simple prompt words into a large model often results in non-compliant generation results due to the ambiguity of the rules, the limitations of the model itself, and potential conflicts between multiple rules, greatly reducing their practical application value in professional fields.

[0004] The maturity of multi-agent system theory and the emergence of general multi-modal large models provide new possibilities for solving the above problems.

[0005] The multi-agent layout generation system based on rule formalization injection proposed by the present application is designed to systematically solve the above problems. The core innovation lies in decoupling the complex layout process into two collaborative subsystems:

[0006] Rule formalization injection process: This process focuses on converting abstract, natural language described design specifications (such as brand manuals, industry standards) into machine executable code level rule constraints through AI model analysis and manual fine-tuning. These formalized rules constitute the factual source of layout quality and compliance.

[0007] Multi-agent layout generation and verification process: This process operates within a human-machine collaboration framework of "human-loop". First, the understanding agent performs in-depth analysis of the user's intent and input materials; then, the layout generation agent quickly generates a complete layout draft based on the analysis results; next, the layout proofreading agent invokes the code-level rule constraints generated in the previous process to perform a rigorous and comprehensive automated check on the draft. If it does not conform to the rules, the system will use a reflection mechanism to feed the problem back to the generation agent for iterative optimization. To prevent infinite loops, the system introduces a checkpoint mechanism, outputting the current best result and archiving it after reaching a certain iteration depth. Users can not only manually modify it to create new checkpoints, but also choose to revert to the previous checkpoint at any time if they are not satisfied with the new version generated by AI, thus achieving a seamless combination between efficient AI generation and precise human control.

[0008] This system is particularly suitable for industry scenarios with extremely high requirements for design standardization, brand consistency, and production efficiency, such as the production of corporate marketing materials, bulk advertising, UI / UX design, and e-commerce product detail page generation. By objectifying subjective standards, making the generation process transparent, and making the modification process traceable, this invention can reduce communication costs, ensure output quality, and improve content production efficiency, possessing practical value and broad commercial promotion prospects. Summary of the Invention

[0009] In view of this, the present invention aims to propose a multi-agent typesetting generation system based on rule-based formal injection, which can reduce communication costs, ensure output quality, and improve content production efficiency.

[0010] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0011] A multi-agent typesetting generation system based on rule-based formal injection includes the following steps:

[0012] Step 1: Rule Formalization: Receive industry regulations and design specifications input in natural language, generate an intelligent agent to parse and transform them, and after manual review and fine-tuning, generate a machine-executable, structured formal rule library for subsequent steps to call;

[0013] Step 2: Requirements and Material Understanding: Receive initial materials and design requirements from the user, or receive an existing intermediate version node with new modification requirements; the VLM multimodal intent understanding agent takes the lead, comprehensively analyzes the input, and distributes specific tasks to downstream professional agents, including: text processing agent, image processing agent, and layout processing agent. They respectively analyze the content, visuals, and structure to form a structured design instruction.

[0014] Step 3: Layout generation and verification cycle:

[0015] a. Overall generation: The layout generation agent receives and integrates structured instructions from multiple upstream processing agents, creating a complete layout plan;

[0016] b. Overall verification: Call the formal rule checking agent, which extracts relevant rules from the rule library in step 1, checks the compliance of the complete layout generated in step a, and outputs the verification result: compliant or non-compliant and specific problems;

[0017] c. Iterative correction: Determine the verification result of step b. If the result is "non-compliant", feedback the problem back to the VLM multi-modal intent understanding agent in step 2, so that it generates new and better structured instructions based on the understanding of the failure reason to guide the next round of regeneration. This cycle can be set to a maximum depth limit. If it still does not comply after reaching the upper limit, output the current best result. If the result is "compliant", the layout meets the requirements and the cycle is terminated;

[0018] Step 4: Version control and human-machine collaboration:

[0019] a. Output and record: The rendering layout agent visually renders the final layout or the best layout output in step 3 and records its state as a version node that can be traced back;

[0020] b. Manual intervention: Users can manually modify and adjust the layout output by the system;

[0021] c. Version rollback: If the user is not satisfied with the AI-generated layout or their own modified layout, the system supports rollback operation, allowing the user to restore to the previous or any specified version node.

[0022] Further, the rule formalization process in step 1 specifically includes:

[0023] S1, natural language input: Support user input of design specifications in text format;

[0024] S2, AI analysis and conversion: The core large language model in the rule formalization generation agent performs semantic analysis on the input natural language rules and converts them into structured and formal rule drafts such as YAML, JSON or specific script languages;

[0025] S3, manual review and storage: Design or technical personnel review, test and accurately adjust the rule draft generated by AI, and store it in the central rule library after confirming that it is correct, for the verification module in step 3 to call at any time.

[0026] Further, its multi-agent collaboration is reflected in:

[0027] Rule formalization generation agent: responsible for converting natural language specifications into machine code in the rule formalization process;

[0028] VLM multi-modal intent understanding agent: as the starting point and overall coordinator of the multi-agent layout generation and verification process, responsible for overall understanding of user needs, and breaking down and distributing raw, unstructured materials and requirements to the following vertical field expert agents for in-depth processing:

[0029] Text processing agent: as an expert in text content, this agent no longer simply transmits text, but performs deep semantic analysis and structured processing, its specific responsibilities include:

[0030] Information level recognition: automatically recognizes and labels main titles, subtitles, body text, quotations, calls to action, and footnotes at different levels;

[0031] Key entity extraction: extracts key information such as dates, times, locations, prices, discounts, names, and product models from text, and tags these entities with special tags for highlighting or special formatting during generation;

[0032] Content summary and expansion: determines whether the text content is too long or too short based on the estimated layout space, and generates summaries or performs appropriate expansion;

[0033] Sentiment and style analysis: analyzes the tone of the text to provide a basis for font selection and layout style in the future;

[0034] Image processing agent: as an expert in visual elements, it performs comprehensive analysis of input image materials, its specific responsibilities include:

[0035] Subject recognition and segmentation: accurately identifies the core subject in the image and can separate it from the background, outputting the subject's mask and contour information;

[0036] Visual saliency analysis: generates a saliency map of the image, highlighting the most eye-catching areas to provide a basis for intelligent cropping and layout emphasis;

[0037] Aesthetic and style evaluation: analyzes the color composition, lighting style, composition type, and aesthetic features of the image;

[0038] Quality evaluation: automatically evaluates the resolution, clarity, and noise level of the image to determine whether it meets the printing or display requirements;

[0039] Layout processing agent: as a space structure planner, this agent is responsible for macro layout planning before formal generation, its specific responsibilities include:

[0040] Content Block Definition: Integrating the outputs of text and image agents, organizing discrete elements into logically related "content blocks";

[0041] Grid System Suggestion: Recommending a suitable grid system as the skeleton of layout, according to content complexity and canvas size;

[0042] Reading Flow Planning: Planning the approximate arrangement order and priority among content blocks based on Z-type and F-type visual dynamic line rules, ensuring the efficiency of information transmission;

[0043] Spatial Relationship Predefinition: Outputting the topological relationships and constraints between elements, rather than specific coordinates;

[0044] Layout Generation Agent: As the final executor, it receives highly structured and information-rich inputs from the above three expert agents, and under these clear instructions and constraints, it collaborates with internal functional modules to complete the final, specific pixel-level layout rendering;

[0045] Verification Agent: As the quality gatekeeper, it strictly checks the generated layout according to the rule base;

[0046] Drawing Layout Agent: Responsible for the final visual presentation, and also records the version control system.

[0047] Further, the formal rule check in step 3 is an overall check, focusing on:

[0048] Receiving a complete, generated layout as input;

[0049] Matching all applicable rules from the rule base to conduct a one-time, comprehensive check on all aspects of the layout;

[0050] Its output is not a numerical score, but a Boolean "compliance" judgment, and an optional list containing descriptions of all violations, which will directly guide the next round of regeneration as feedback information.

[0051] Further, the iterative correction mechanism in step 3 has depth limits and intelligent reflection:

[0052] The system records the number of iterations of the current layout generation, i.e., the number of "non-compliance -> regeneration" cycles; when the number of iterations exceeds the preset upper limit of depth, even if the current layout is still not completely checked, the system will terminate the cycle and output the "closest to compliance" layout result in the current round;

[0053] Its correction cycle is not a simple random retry, but the violation feedback is returned to the understanding layer, so that the system can "know and correct" in the next generation, thereby improving the iteration efficiency.

[0054] Further, the ability of the page generation agent includes:

[0055] Font selection and style application, brand color plate compliance correction, element alignment and distribution, safety margin and white space control, grid system application, picture intelligent cropping and layout, text block automatic line breaking and alignment, information level visual expression, background intelligent generation and replacement, element position and size adjustment, overall composition balance optimization function module;

[0056] The page generation agent cooperates with the calling of these internal modules to output a complete page according to the structured input provided by the upstream text, image and layout processing intelligent agent. BRIEF DESCRIPTION OF DRAWINGS

[0057] Fig. 1 is the overall flowchart of the multi-agent layout generation system based on rule formalization injection according to the present application;

[0058] Fig. 2 is a rule formalization flowchart of the multi-agent layout generation system based on rule formalization injection according to the present application;

[0059] Fig. 3 is a multi-agent page generation and verification cycle diagram of the multi-agent layout generation system based on rule formalization injection according to the present application;

[0060] Fig. 4 is a version control and man-machine cooperation flowchart of the multi-agent layout generation system based on rule formalization injection according to the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings, such as Figs. 1 to 4 It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

[0062] The present application provides a multi-agent layout generation system based on rule formalization injection, comprising the following steps:

[0063] Step 1: Rule formalization: this step aims to convert the industry regulations or design specifications expressed in natural language and fuzzy into structured instructions that can be accurately executed by machines, which is the basis for all subsequent automatic generation and verification.

[0064] In one embodiment, this step specifically includes:

[0065] S1. Natural Language Rule Input:

[0066] The system provides an interface that allows authorized users (such as design supervisors or technicians) to input rules in textual form.

[0067] For example: Input a brand marketing specification: "Call-to-Action (CTA) elements must be the most visually prominent elements on the page, only second to the main title." Or input a content compliance rule: "In financial product promotions, the font size of risk warnings must not be less than 80% of the body text."

[0068] S2. AI Analysis and Conversion:

[0069] The rule formalization generates an intelligent agent that receives the above natural language input. The intelligent agent has a large language model integrated inside, which has been trained to perform semantic analysis, intent recognition, and key parameter extraction on the input text. The model converts the analysis results into a preset structured data format, such as JSON, YAML, or XML.

[0070] For example: The intelligent agent decomposes the abstract concept of "visual prominence" into multiple quantifiable and verifiable dimensions, and generates a formal representation object.

[0071] S3. Manual Review and Storage:

[0072] The structured rule draft generated by AI is presented to authorized users for review. Users can fine-tune, confirm, or veto the parameters (such as the value of the minimum frame size). After the review is passed, the structured rule is officially stored in a central formal rule library for subsequent verification agents to call.

[0073] Step 2: Understanding Requirements and Materials: This step receives the design requests and original materials of the end user, and through a multi-agent collaboration system, it converts unstructured input into a highly structured design instruction for guiding page generation. In an embodiment, when the user inputs the initial materials (such as images, text files) and design requirements (such as "make a promotional poster"), the system processes as follows:

[0074] The VLM multi-modal intent understanding agent acts as the leading agent, first conducting a comprehensive analysis of user requirements and materials.

[0075] It identifies core design goals such as "promotion", style preferences such as "vibrant" or "business", and distributes specific types of materials to downstream specialized agents.

[0076] Parallel processing of downstream specialized agents:

[0077] The text processing agent analyzes the content of the text and outputs structured data, including but not limited to information hierarchy (such as title h1, body p), key entities (such as price, date), and sentiment style.

[0078] The image processing agent analyzes the image material and outputs structured information including subject mask, visual saliency map, color composition, and quality assessment.

[0079] The layout processing agent integrates the outputs of the above agents, conducts macro layout planning, and defines the topological relationship and constraint conditions between content blocks, such as "Logo block must be above the title block" and "Product image and price block must be adjacent".

[0080] Step 3: Page generation and verification loop: This step is the core execution link of the system, which automatically creates a page that meets the formal rules in an iterative manner through a "generation-verification-feedback-correction" closed loop mechanism.

[0081] In one embodiment, the loop process is as follows:

[0082] a. Overall generation: The page generation agent receives and integrates the structured instructions from multiple agents in step 2. It generates a preliminary draft of a complete page layout scheme according to these instructions by calling internal typesetting function modules (such as alignment, distribution, font application, etc.).

[0083] In the first round of generation, the agent selected a light gray background (#CCCCCC) for the "grab immediately" button, resulting in a contrast ratio of only 1.2 with the white page background.

[0084] b. Overall verification: The formal rule checking agent is activated. It extracts all applicable rules from the rule library established in step 1 and performs a one-time, comprehensive compliance scan of the generated complete page. Its characteristic is that the output of this agent is a boolean "compliance" judgment and an optional list containing descriptions of all violation points.

[0085] For example, for the above generated page, the verification agent loads the JSON rules.

[0086] The verification agent will check each object in the array like executing a unit test suite:

[0087] It calls the color contrast module, passing in the page data and color variable parameters for calculation.

[0088] It calls the relative size module, passing in the page data and relative size parameters for comparison.

[0089] All checks are performed sequentially. Suppose in the checks, it is found that the white space around the CTA button is insufficient, not meeting the white space size check. The formal rule check agent will immediately generate an accurate, structured violation feedback.

[0090] This output does not provide a numerical score, but gives an accurate, executable error report.

[0091] c. Iterative correction: The system judges the verification result of step b. If conflicts are detected, the information in the conflict list is fed back to the VLM multi-modal intent understanding agent in step 2.

[0092] Its characteristic is that the correction cycle is not simply a random retry. It can clearly understand "which element, which attribute, what is the current value, what is the required value". Based on this, the system can make targeted corrections in the next iteration (for example, only increase the outer spacing of elements with a specific ID), rather than vague and random retries.

[0093] The layout generation agent, after receiving the correction instructions, performs the second round of generation, at which time it will ensure that only the error part is modified.

[0094] This cycle is repeated until the verification result is true, or the preset upper limit of iteration depth (for example, 10 times) is reached. If the upper limit is reached and it is still not compliant, the system outputs the version with the least violations or the least serious problems in the current cycle as the "current best" result.

[0095] Step 4: Version control and human-machine collaboration: This step provides management capabilities for system output and retains a channel for human designers to make final adjustments and decisions, achieving an efficient human-machine collaborative workflow. In one embodiment, this step specifically includes:

[0096] a. Output and record: The final compliant layout (or best layout) output in step 3 is visually rendered by the layout drawing agent to generate the final design file (such as PNG, PDF). At the same time, all state parameters of this layout are packaged and recorded as a stable version node in the version control system.

[0097] b. Manual intervention: Users can use the accompanying editor to make manual fine-tuning adjustments based on the AI-generated layout. For example, the designer may think that the AI-generated title position can be moved up 5 pixels for better visual balance. Each manual save creates a new version node in the version control system.

[0098] c. Version rollback: the system provides version history. If the user is not satisfied with the subsequent AI-generated version or his own manual modification, he can restore to any previous saved version node through rollback operation, thereby ensuring the safety and flexibility of the design process.

[0099] In summary, the present application constructs an automatic typesetting generation system with self-checking and iterative correction capabilities through rule formalization injection and multi-agent collaboration, and realizes efficient human-computer collaboration through version control mechanism, thereby improving the efficiency and standardization of design and production.

[0100] Those skilled in the art should understand that the above description is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A multi-agent typesetting generation system based on rule-based formal injection, characterized in that, The system includes: Rule-formal generation of intelligent agents: used in process one, responsible for converting natural language specifications into machine code, where process one is a rule-formalization process; The VLM multimodal intent understanding agent, acting as the starting point and overall coordinator of process two, is responsible for comprehensively understanding user needs and decomposing and distributing the raw, unstructured materials and requirements to downstream specialized agents for in-depth processing. Process two is the multi-agent layout generation and verification process. Text processing agent: As an expert in text content, this agent no longer merely transmits text, but performs deep semantic analysis and structured processing. Its specific responsibilities include: Information hierarchy recognition: Automatically identifies and marks different levels of text, such as main title, subtitle, body text, citations, calls to action, and footnotes; Key entity extraction: Extract key information from the text, such as date, time, location, price, discount, name, and product model, and label these entities with special tags for highlighting or special formatting during generation; Content summarization and expansion: Based on the estimated layout space, determine whether the text content is too long or too short, and generate a summary or expand it; Sentiment and style analysis: Analyzing the tone of the text to provide a basis for subsequent font selection and layout style; Image processing agent: As an expert in visual elements, it performs comprehensive analysis of input image materials. Its specific responsibilities include: Subject recognition and segmentation: accurately identifies the core subject in an image and separates it from the background, outputting the subject's mask and contour information; Visual saliency analysis: Generates a saliency map of the image, marking the most eye-catching areas to provide a basis for intelligent cropping and layout emphasis; Aesthetic and style assessment: Analyze the aesthetic characteristics of the image's color composition, lighting style, and composition type; Quality assessment: Automatically assesses the resolution, sharpness, and noise level of an image to determine whether it meets printing or display requirements; Layout processing agent: As a planner of spatial structure, this agent is responsible for macro-level layout planning before formal generation. Its specific responsibilities include: Content block definition: Integrating the output of text and image agents, the discrete elements are organized into logically related "content blocks"; Grid system recommendation: Based on the complexity of the content and the canvas size, a suitable grid system is recommended as the framework of the layout; Reading flow planning: Based on the Z-shaped and F-shaped visual flow patterns, plan the arrangement order and priority between content blocks to ensure the efficiency of information transmission; Predefined spatial relationships: Output the topological relationships and constraints between elements, rather than specific coordinates; Page layout generation agent: As the final executor, it receives highly structured and information-rich input from the aforementioned text processing agent, image processing agent, and layout processing agent, and under these clear instructions and constraints, it coordinates and calls various internal functional modules to complete the final, specific pixel-level page layout rendering; Formal rule checking agent: Acts as a quality gatekeeper, strictly checking the generated products according to the rule base; The layout agent is responsible for the final visual presentation and also handles the recording work of the version control system. The system performs the following steps: Step 1: Rule Formalization: Receive industry regulations and design specifications input in natural language, generate an intelligent agent to parse and transform them, and after manual review and fine-tuning, generate a machine-executable, structured formal rule library for subsequent steps to call; Step 2: Requirements and Material Understanding: Receive initial materials and design requirements from the user, or receive an existing intermediate version node with new modification requirements; the VLM multimodal intent understanding agent takes the lead, comprehensively analyzes the input, and distributes specific tasks to downstream professional agents, including: text processing agent, image processing agent, and layout processing agent. They respectively analyze the content, visuals, and structure to form a structured design instruction. Step 3: Page Layout Generation and Verification Loop a. Overall Generation: The layout generation agent receives and integrates structured instructions from multiple upstream processing agents to create a complete layout scheme; b. Overall verification: Call the formal rule check agent, extract relevant rules from the rule base in step 1, perform compliance verification on the complete layout generated in step a, and output the verification result: compliant or non-compliant and the specific issues of non-compliance; c. Iterative Correction: Determine the verification result of step b. If the result is "non-compliant", the problem feedback is returned to the VLM multimodal intent understanding agent in step 2, so that it can generate new and better structured instructions based on understanding the reason for failure, to guide the next round of regeneration; this loop can be set with a maximum depth limit. If it is still non-compliant after reaching the limit, the current best result is output; if the result is "compliant", the layout is determined to meet the requirements and the loop terminates. Step 4: Version Control and Human-Computer Collaboration a. Output and Recording: The layout drawing agent performs visual rendering on the final or best layout output in step 3 and records its state as a version node that can be traced back; b. Manual intervention: Users can manually and finely modify and adjust the layout output by the system; c. Version rollback: If a user is not satisfied with the layout generated by AI or the layout they modified, the system supports rollback, allowing the user to restore to the previous or any specified version node.

2. The multi-agent typesetting generation system based on rule-based formal injection according to claim 1, characterized in that, The rule formalization process in step 1 specifically includes: S1. Natural Language Input: Design specifications that support user-input text formatting; S2, AI Parsing and Transformation: The core large-scale language model in the intelligent body performs semantic analysis on the input natural language rules and transforms them into structured and formal rule drafts in YAML, JSON, or specific scripting languages; S3. Manual review and storage: Designers or technicians review, test and fine-tune the AI-generated rule drafts, and store them in the central rule library after confirming that they are correct, so that they can be called up at any time in step 3.

3. The multi-agent typesetting generation system based on rule-based formal injection according to claim 1, characterized in that, The formal rule check in step 3 is a holistic verification process, and its capabilities focus on: It accepts a complete, pre-generated page layout as input; Match all applicable rules from the rule base to perform a one-time, comprehensive check on all aspects of the layout; Its output is not a numerical score, but a Boolean "compliance" judgment, and an optional list containing descriptions of all violations, which will be used as feedback to guide the next round of regeneration.

4. The multi-agent typesetting generation system based on rule-based formal injection according to claim 1, characterized in that, The iterative correction mechanism in step 3 has depth limitations and intelligent reflection: The system will record the number of iterations generated for the current layout, that is, the number of loops of "non-compliant -> regenerate". When the number of iterations exceeds the preset depth limit, even if the current layout has not completely passed the verification, the system will terminate the loop and output the layout result that is "closest to compliance" in the current round. Its correction loop is not a simple random retry, but rather returns violation feedback to the understanding layer, allowing the system to "correct its mistakes" in the next generation, thereby improving iteration efficiency.

5. The multi-agent typesetting generation system based on rule-based formal injection according to claim 1, characterized in that, The page layout generating agent includes: Modules include: font selection and style application, brand color palette compliance correction, element alignment and distribution, safety margin and white space control, grid system application, intelligent image cropping and layout, automatic text wrapping and alignment, information hierarchy visual expression, intelligent background generation and replacement, element position and size adjustment, and overall composition balance optimization. The layout generation agent, based on the structured input provided by the upstream text, image, and layout processing agents, collaboratively calls on the following functional modules to produce a complete layout: font selection and style application, brand color palette compliance correction, element alignment and distribution, safety margin and white space control, grid system application, intelligent image cropping and layout, automatic text block wrapping and alignment, information hierarchy visual expression, intelligent background generation and replacement, element position and size adjustment, and overall composition balance optimization.

Images