Information processing device, information processing method, and program

Abstract

To provide an information processing device, an information processing method, and a program that can automatically generate a general-purpose template capable of handling dynamic data from a single visual design sample. [Solution] An information processing device comprising: an acquisition unit that acquires sample data showing the layout of a design; a first generation unit that generates a first code that describes the arrangement and style of elements included in the design using fixed values ​​based on the sample data; and a second generation unit that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​included in the first code into variables or control structures.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program. Background Art

[0002] Conventionally, techniques for automatically generating designs for presentation materials and the like are known. For example, Patent Document 1 discloses a technique for automatically generating a design that promotes understanding of text content by classifying input text and matching a diagram illustration according to the content of the text. Prior Art Documents Patent Documents

[0003] Patent Document 1 Japanese Unexamined Patent Publication No. 2025-035448 Summary of the Invention Problems to be Solved by the Invention

[0004] The technique described in Patent Document 1 stores a design template for each of a plurality of predefined classifications, selects a design template according to input text, and generates a design. In such a case, engineers need to manually hard-code coordinates and colors for the design templates, which requires an enormous amount of man-hours to prepare. Furthermore, manually created templates have low flexibility against fluctuations in input dynamic content (such as text strings and the number of characters), and there has been a problem that the layout is easily broken.

[0005] The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an information processing apparatus, an information processing method, and a program capable of automatically generating a general-purpose template compatible with dynamic data from a single visual design sample. Means for Solving the Problems

[0006] One aspect of the present invention is an information processing device comprising: an acquisition unit that acquires sample data showing the layout of a design; a first generation unit that generates a first code that describes the arrangement and style of elements included in the design using fixed values ​​based on the sample data; and a second generation unit that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​included in the first code into variables or control structures. [Effects of the Invention]

[0007] According to one aspect of the present invention, an information processing device, an information processing method, and a program can be provided that can automatically generate a general-purpose template capable of handling dynamic data from a single visual design sample. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of System 1, which includes the information processing device 100 according to this embodiment. [Figure 2] This diagram shows an overview of the processing in the first information processing unit 110. [Figure 3] This figure shows an example of a prompt 134 used by the first generation unit 114 when generating the hardcode 136. [Figure 4] This figure shows an example of a prompt 134 used by the second generation unit 116 when generating the template function 138 and the configuration data 140. [Figure 5] This flowchart shows an example of the processing flow executed by the first information processing unit 110. [Figure 6] This figure shows an overview of the processing in the second information processing unit 120. [Figure 7] This figure shows an example of a prompt 134 used by the evaluation unit 126 when generating evaluation results. [Figure 8] This figure shows an example of a prompt 134 used by the modification unit 128 when modifying the setting data 140. [Figure 9] This figure shows an example of an arithmetic correction made by the correction unit 128. [Figure 10] This flowchart shows an example of the processing flow performed by the second information processing unit 120. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of the information processing apparatus, information processing method, and program of the present invention will be described with reference to the drawings.

[0010] [Challenges in the underlying technology] In recent years, there has been a growing need among companies and other organizations to automatically generate presentation materials and designs that conform to brand style and guidelines. However, with conventional technology, creating design templates required engineers to manually measure the coordinates, colors, font sizes, etc., of elements and hardcode them into the program code, resulting in enormous man-hours and costs for construction and maintenance.

[0011] Furthermore, these manually created static template engines had the problem of low flexibility in dealing with variations in dynamic input content because they relied on predefined placeholders. For example, even slight changes in data, such as the input text being longer than expected or the number of items in a list increasing or decreasing, could easily cause the layout to break. In addition, there are significant challenges in quality assurance (QA) of materials containing dynamic content. When automatically verifying whether the generated material has the correct layout, the text content changes each time, so traditional image regression testing (Visual Regression Testing), which compares the correct image pixel by pixel, may fail (false positive) due to differences in text even if the layout is correct.

[0012] [Summary of the Embodiment] In order to solve such problems, the information processing apparatus 100 according to the present embodiment provides a mechanism based on the concept of "Automated Design Factory". The system of the present embodiment is broadly divided into two technical functions: a first information processing unit 110 and a second information processing unit 120.

[0013] The first function (the first information processing unit 110) functions as a "style-deterministic code compiler" that automatically generates a general-purpose template capable of supporting dynamic data from a single visual design sample. With this function, only by inputting a design sample, an AI learns its layout and brand rules through reverse engineering, and automatically compiles a deterministic drawing script that produces the same result regardless of who executes it (for example, an automated scripting language that operates in a cloud environment, or a general-purpose programming language).

[0014] The second function (the second information processing unit 120) functions as a "visual language model-guided visual validator" in which an AI automatically verifies and automatically corrects the quality of generated code. With this function, test drawing is performed using the generated code, and a visual language model (VLM) visually compares deviations from the original sample (correct answer). If there is a style deviation, the visual feedback is mapped to the logic of the source code, and a self-repair loop that automatically applies patches is executed.

[0015] By linking these two functions, the information processing apparatus 100 automatically constructs a system with dynamic generalization capability capable of supporting unknown data from a visual sample, and furthermore, the system itself repairs unexpected defects (hallucinations) such as layout collapse of generated products, thereby realizing automatic operation that does not require human intervention (with low maintenance costs).

[0016] [System Configuration] FIG. 1 is a diagram showing an example of the overall configuration of a system 1 including an information processing apparatus 100 according to the present embodiment. This system includes the information processing apparatus 100 and an AI server 200, which are communicably connected to each other via a network NW. The network NW is any communication network, for example, the Internet, a LAN (Local Area Network), a dedicated line, or the like.

[0017] The information processing apparatus 100 is implemented by, for example, a server computer including a CPU (Central Processing Unit) and a large-capacity memory. The information processing apparatus 100 includes, as functional components, a first information processing unit 110, a second information processing unit 120, and a storage unit 130. These components are implemented by, for example, a hardware processor such as a CPU included in the information processing apparatus 100 reading and executing a program (software) stored in the storage unit 130. Some or all of these components may be implemented by dedicated hardware including circuitry such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by cooperation of software and hardware.

[0018] Although FIG. 1 shows a configuration in which the first information processing unit 110 and the second information processing unit 120 are included in the same information processing apparatus 100, the configuration is not limited thereto. Alternatively, the first information processing unit 110 and the second information processing unit 120 may be distributed and arranged on different physical or logical servers. For example, a flexible configuration can be adopted according to the processing load, such that the first information processing unit 110 functions as a development / compilation server and the second information processing unit 120 independently functions as a test / quality assurance server.

[0019] The AI ​​server 200 is a server device or cloud service that performs multimodal inference processing and hosts a Visual-Language Model (VLM) 210 and a Language Model (LLM) 220. The VLM 210 and LLM 220 may be physically different models, or alternatively, they may be configured as a single multimodal platform model capable of integrating and processing both visual and textual information.

[0020] VLM210 is a pre-trained model that accepts both image and text data as input and can recognize and analyze the visual features of images (layout, coordinates, element overlaps, typography, etc.) like the human eye and output the results. LLM220 is a large-scale language model that accepts text (including program code) as input and performs code analysis, refactoring, and logical reasoning (patching, etc.). Alternatively, LLM220 may be a model of a code repair engine (such as a code generation agent) that is finely tuned specifically for the task of writing and modifying program code.

[0021] The memory unit 130 is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and it temporarily or permanently stores various data necessary for processing. For example, the memory unit 130 stores sample data 132, various prompts 134 which are instruction information for the AI ​​model, hardcoded data 136, template functions 138, and configuration data 140.

[0022] The first information processing unit 110 is a module that implements the "style-deterministic code compiler" function described above, and comprises an acquisition unit 112, a first generation unit 114, and a second generation unit 116. The second information processing unit 120 is a module that implements the "VLM-guided visual validator" function described above, and comprises an acquisition unit 122, an execution unit 124, an evaluation unit 126, and a modification unit 128.

[0023] [Detailed processing of the first information processing unit] Figure 2 shows an overview of the processing in the first information processing unit 110. The first information processing unit 110 executes a three-stage process to automatically generate general-purpose template code from visual sample data 132.

[0024] In this embodiment, instead of directly converting code from image input using a single AI model, a two-stage process is performed: "perfect reproduction of what is seen" using VLM210 and "abstraction of rules" using LLM220. This is to clearly distinguish between "fixed rules (configurations)" such as brand colors and margins, and "variable rules" such as the amount of text and the number of items in a list, in order to generate highly accurate general-purpose templates.

[0025] First, the acquisition unit 112 acquires sample data 132 that shows the layout of the design. The sample data 132 is, for example, a single presentation slide, an electronic document file, or an image file (JPEG or PNG format, etc.). Alternatively, the sample data 132 may be a hand-drawn mockup image or a screenshot of a web design, etc.

[0026] Here, if the sample data 132 is in a file format with an internal structure such as presentation slide data, the acquisition unit 112 or the first generation unit 114 may use a dedicated file parser or the like to extract structured data from the file that represents the geometric and stylistic properties of the elements. The structured data may be a JSON-formatted payload that includes, for example, the type of each element, the coordinates of the bounding box, dimensions, hexadecimal color code specification, string representation, etc. Alternatively, this structured data may be in XML format, YAML format, or the like. By using this structured data in addition to, or instead of, image input, accurate positional information that does not rely on inference from pixels can be provided to subsequent processing, and the accuracy of generalization can be greatly improved.

[0027] Next, the first generation unit 114 generates a first code (hardcode 136 as a seed function) that describes the arrangement and style of the elements included in the design using fixed values, based on the sample data 132. At this time, the first generation unit 114 visually analyzes the images, etc., of the sample data 132 using the VLM210.

[0028] Figure 3 shows an example of a prompt 134 used by the first generation unit 114 when generating hardcode 136. The prompt 134 shown in Figure 3 functions as a prompt for a deterministic translation engine. The prompt 134 shown in Figure 3 includes, for example, region A1 (#input data), region A2 (#execution task), and region A3 (#execution constraints).

[0029] Area A1 is input with the image from sample data 132 and / or structured data (such as a JSON payload) extracted from sample data 132. The target output programming language and the target rendering API (Application Programming Interface) are also specified. Alternatively, the target language may be a specific scripting language or macro language, and the target API may be an API for manipulating the DOM (Document Object Model) of a web browser or a presentation object model of a specific office software.

[0030] In area A2, you are instructed to translate the input data exactly one-to-one into executable drawing code (sequential hardcoded) for the target API.

[0031] In area A3, three strict rules are defined that determine the output characteristics of the first generation unit 114. The first constraint is "zero abstraction." This instructs against creating any variable definitions, configuration objects, or logical iterative processes (such as logical loops).

[0032] The second constraint is "Use of Absolute Values." Absolute coordinates (X, Y) and style values ​​(color codes, dimensions, etc.) defined in the input are instructed to be hardcoded as fixed values ​​(literal values). These values ​​may include string literals and typographic information such as font properties.

[0033] The third constraint is "Command Mapping." Each element of the sample data 132 is instructed to be directly mapped to the corresponding native API drawing method (e.g., inserting shapes, creating text boxes, styling attributes, etc.). The output format may also be instructed to return only executable function blocks, without including interactive text or explanations. In this way, by completely eliminating the logical complexity of programming, VLM210 can specialize solely in "reproducing visual accuracy" and generate hardcoded 136 without layout issues.

[0034] Returning to Figure 2, the second generation unit 116 analyzes the hardcode 136 using LLM220, identifies repeating patterns, and injects variables to generalize (refactor) the dynamic data into a general-purpose second code (template function 138) that can be rendered. At the same time, the second generation unit 116 extracts and separates style-related constants as independent configuration data 140.

[0035] Figure 4 shows an example of a prompt 134 used by the second generation unit 116 when generating the template function 138 and configuration data 140. The prompt 134 shown in Figure 4 functions as a prompt for the software architect and template compiler. The prompt 134 shown in Figure 4 includes, for example, area A4 (#input data), area A5 (#execution task), and area A6 (#execution constraints).

[0036] Area A4 receives the hardcode 136 output by the first generation unit 114. Furthermore, to dramatically improve the accuracy of generalization, the second generation unit 116 may input at least one of the original sample data 132 image or structured data obtained from the sample data 132 as additional information to the LLM220. This allows the LLM220 to accurately grasp the original visual intent, not just from a mere code string. Additionally, natural language prompts indicating the developer's intent may be included as additional information.

[0037] In area A5, the hardcode 136 is instructed to be refactored into a general-purpose template function (template function 138) capable of rendering dynamic data, and further, a configuration file (configuration data 140) representing the hardcode style is to be separated and output.

[0038] In area A6, three abstraction rules for generalization are defined. The first constraint is "Style Extraction." Constants such as color, coordinates, and margins are identified and extracted from the hardcoded 136 and separated as a dictionary or data object of independent configuration data 140 (Config). Fixed values ​​in template functions 138 are replaced with references to this configuration data 140. This ensures that brand design rules are managed in one place and consistency is guaranteed.

[0039] The second constraint is "Variable Injection." This involves identifying fixed text strings and image URIs within hardcoded elements and replacing them with dynamic variables (placeholders) referenced by the external payload (the dynamic data being input).

[0040] The third constraint is "loop generation." Hardcode 136 identifies a sequence of commands that repeat in a predetermined pattern with spatial offsets (for example, list items drawn with consistent offsets), and converts that redundant block into an iterative control structure (such as a logical loop structure) that is appropriate to the length of the data array.

[0041] For example, suppose the input data 132 is a flyer design with five restaurant menus for November arranged at regular intervals. The first generation unit 114 hardcodes this as five independent elements using absolute values. Then, the second generation unit 116 discovers the spatial regularity that these five elements are arranged at equal intervals and refactors it into a loop construct that dynamically calculates the coordinates using the number of menu items as the length of the data array.

[0042] As a result, even if dynamic data is entered the following month with an increased number of menu items (7), the generated template function 138 automatically understands the number of elements in the variable and can automatically generate a layout with the 7 menu items properly aligned without any problems, while maintaining the same style rules as the original case with 5 items. Alternatively, the target is not limited to menus; for example, it could be the structure of a calendar, an organizational chart, a product catalog, etc.

[0043] Figure 5 is a flowchart showing an example of the processing flow executed by the first information processing unit 110. The processing shown in the flowchart in Figure 5 is executed, for example, when the administrator of the information processing device 100 stores the sample data 132 in the storage unit 130 and gives an execution instruction (for example, an execution button) to the first information processing unit 110.

[0044] In step S100, the acquisition unit 112 acquires sample data 132 that shows the layout of the design. In step S102, the first generation unit 114 analyzes the image of the sample data 132 using VLM210 or the like, and generates hardcode 136 that describes the arrangement and style of the elements included in the design using fixed values.

[0045] In step S104, the second generation unit 116 analyzes the hardcode 136 using LLM220 and generates a generalized template function 138 and configuration data 140 by converting at least some of the fixed values ​​into variables and control structures.

[0046] In step S106, the generated template function 138 and setting data 140 are stored in the storage unit 130 and saved as a deliverable that can be verified by the second information processing unit 120, which will be described later. With this, the processing of the flowchart shown in Figure 5 is completed.

[0047] [Detailed processing of the second information processing unit] Figure 6 shows an overview of the processing in the second information processing unit 120. The second information processing unit 120 verifies whether layout issues occur when dynamic content is provided to the generated template function 138, and executes a closed-loop QA (Quality Assurance) cycle that automatically corrects the source code if an error is found.

[0048] First, the execution unit 124 executes the program code (template function 138 and setting data 140) generated by the first information processing unit 110 to temporarily render a test "candidate image (candidate slide)".

[0049] Next, the acquisition unit 122 acquires the generated candidate image and the reference image (original sample data 132, etc.) which is the correct design (oracle). Then, the evaluation unit 126 (VLM Judge) uses VLM 210 as the "evaluator" to compare the candidate image and the reference image and evaluate the differences in style. By freeing human reviewers from visual inspection and replacing them with VLM that compares output and input, the review process can be completely automated.

[0050] Figure 7 shows an example of a prompt 134 used by the evaluation unit 126 when generating evaluation results. The prompt 134 shown in Figure 7 functions as a prompt for the automated visual QA oracle. The prompt 134 shown in Figure 7 includes, for example, area A7 (#input data), area A8 (#execution task), and area A9 (#execution constraints).

[0051] Area A7 (#Input Data) contains the correct reference image and the candidate image to be validated. Area A8 (#Execution Task) instructs the user to compare and evaluate the structural and stylistic consistency between the two images.

[0052] In area A9 (#Execution Constraints), two conditions for verification are specified. The first constraint is "Content Agnostic". The evaluation unit 126 is strictly instructed to "completely ignore" the actual character content of text, the length of strings, or differences in the subject matter of images, and to focus only on layout, bounding boxes, and typography. This prevents false positives in pixel difference regression tests caused by content variations in dynamic data.

[0053] The second constraint is "Style Verification." It is instructed to extract and verify only differences in spatial arrangement (alignment of X / Y coordinates), margins (padding), font attributes (weight, etc.), color, and element stacking order (Z index, layer, etc.). While rule-based analysis tools require explicit rule descriptions, VLM210 of this technology infers these implicit rules from the reference image and performs the verification.

[0054] The evaluation unit 126 outputs this evaluation result as an error report in a structured data format (such as JSON). Alternatively, the evaluation result may be in another format that is easy for a program to parse, such as XML. For example, if the styles match perfectly, it simply outputs a status of {"status": "PASS"}.

[0055] If differences exist, the output will be {"status": "FAIL", "errors": []}, and will include information identifying the "target element" in the array where the visual style difference was detected (e.g., "target": "subtitle"), the nature of the spatial or stylistic difference in that target element, and the "estimated difference value of dimensions (in units such as pixels) or hexadecimal color code" (e.g., "error": "Subtitle element Y-coordinate is ~15 units too high").

[0056] Returning to Figure 6, the patching agent (Patching Agent) receives the erroneous evaluation results and functions as a "code repair engine" that uses the LLM220 to self-repair the style configuration data 140 referenced by the program code.

[0057] Figure 8 shows an example of a prompt 134 used by the correction unit 128 when correcting the configuration data 140. The prompt 134 shown in Figure 8 functions as a prompt for the code repair engine. The prompt 134 shown in Figure 8 includes, for example, area A10 (#input data), area A11 (#execution task), and area A12 (#execution constraint).

[0058] Area A10 (#Input Data) receives the current configuration data 140 and the evaluation results (error report) output from the evaluation unit 126 mentioned above. Area A11 (#Execution Task) is instructed to automatically correct the visual discrepancies by mapping them to the variables in the configuration data 140.

[0059] Area A12 (#Execution Constraints) specifies the concrete correction procedure. First, interpret the visual discrepancy reported as an error (e.g., "Y coordinate is approximately 15 units too high"). Second, as variable identification, identify the corresponding placement / style variable (e.g., "subtitle_start_y") from the configuration data.

[0060] Thirdly, as an arithmetic or stylistic correction, mathematical corrections (e.g., subtraction or addition from the current value) or direct substitution of values ​​are applied to the identified variable values ​​to resolve the discrepancies, and the updated configuration data is output.

[0061] Figure 9 shows an example of arithmetic correction by the correction unit 128. First, let's assume that the evaluation result (error report) output from the evaluation unit 126 gives the feedback ""target": "title"" ""error": "Y coordinate is 15 units too high". At the same time, let's assume that the constant for the corresponding title position in the current setting data 140 is "const title_start_y = 100;".

[0062] The modification unit 128 links these together and performs an arithmetic subtraction of "100-15=85" to lower the position. Alternatively, the modification unit 128 may perform a relative ratio change or a direct replacement of the color code. As a result, "const title_start_y = 85;" is output as the new configuration data 140. The output format may be instructed to return only the updated data object without including explanatory text, etc. This automates the cycle from code modification to re-execution, which was previously performed manually by a human.

[0063] Figure 10 is a flowchart showing an example of the processing flow performed by the second information processing unit 120. The processing shown in the flowchart in Figure 10 may be executed, for example, when the administrator of the information processing device 100 gives an execution instruction (for example, an execution button) to the second information processing unit 120, or it may be executed automatically when the processing by the first information processing unit 110 is completed.

[0064] In step S200, the execution unit 124 executes program code based on the template function 138 and the current setting data 140 to generate candidate images. In step S202, the evaluation unit 126 uses the VLM 210 to evaluate the style differences between the candidate images and the reference images.

[0065] In step S204, based on the evaluation results, it is determined whether or not there are style differences. If there are no differences (status is PASS) (S204:NO), the template code is certified as valid, and the process proceeds to S208.

[0066] If there is a discrepancy (status is FAIL) (S204:YES), step S206 determines whether the current number of loops is greater than or equal to a pre-set upper limit (for example, 3 or 5 times, or a value that can be specified by the user). If it is greater than or equal to the upper limit (S206:YES), the process proceeds to S208 to prevent an infinite loop.

[0067] If the number of loop iterations is less than the upper limit (S206: NO), in step S210, the modification unit 128 modifies the setting data by applying arithmetic corrections or value substitutions to the variable values ​​in the setting data 140 based on the evaluation results.

[0068] In step S212, the execution unit 124 executes the code again using the corrected configuration data 140 to generate new candidate images. The process returns to S202, and this self-repair loop is executed iteratively until no further differences in visual style are detected. In step S208, the final configuration data 140, which has been determined to have no differences (or has reached the upper limit), is output or saved, completing the series of processing steps.

[0069] [Application Fields] The technology according to this embodiment can be applied to various business uses. One application is its deployment as an in-house tool. By building a design factory environment within the company where anyone can obtain automated scripts from mockups created by designers, company-wide document creation can be streamlined. Alternatively, the end user may simply input instructions in natural language, and an AI agent will interpret those instructions and call a generated template function to complete high-quality documents in seconds.

[0070] Another application area is its deployment in B2B products. For example, customers can simply upload electronic documents defining their brand rules and samples to the system, and a document generation environment compliant with their brand can be instantly deployed as a product function.

[0071] [summary] As described above, the information processing device 100 according to this embodiment includes an acquisition unit 112 that acquires sample data 132 showing the layout of a design, a first generation unit 114 that generates a first code (hardcode 136) that describes the arrangement and style of elements using fixed values ​​based on the sample data, and a second generation unit 116 that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​into variables or control structures.

[0072] This allows for the automatic generation of a general-purpose template capable of handling dynamic data from a single visual design sample. The first code is written using absolute values ​​without including variable definitions or logical iteration, thus generating a seed function that is faithful to the visual design sample. Furthermore, the second generation unit 116 extracts style constants and separates and generates them as setting data 140, and by performing variable injection and loop generation, it can flexibly respond to changes in dynamic content.

[0073] Furthermore, the first generation unit 114 can handle a variety of input formats with high accuracy because, if the sample data 132 includes structured data, it directly maps it to API instructions, and if it includes images, it performs visual analysis using a visual language model. In addition, the second generation unit 116 can further improve generalization accuracy by inputting images and structured data as additional information to the language model in addition to the first code.

[0074] Furthermore, the information processing device 100 includes an acquisition unit 122 that acquires candidate images generated using a second code and a correct reference image, an evaluation unit 126 that compares these using a visual language model and evaluates the differences in visual style, and a modification unit 128 that modifies the setting data 140 based on the evaluation results.

[0075] The evaluation unit 126 completely ignores differences in text content and subject matter, and verifies only differences in style such as spatial arrangement and font attributes. This prevents regression test failures specific to dynamic data. Furthermore, since the evaluation results are output as structured data including target elements and difference values, direct feedback to the program is possible.

[0076] The correction unit 128 identifies the corresponding constants within the configuration data 140 and applies arithmetic corrections or value substitutions. The execution, evaluation, and correction processes are then executed iteratively until no style differences are detected. This closed-loop system allows the system to autonomously detect and self-repair bugs, achieving maintenance-free automated operation.

[0077] [Differentiation] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0078] For example, although this embodiment primarily describes the automatic generation of presentation materials, the scope of the present invention is not limited to this. It can be broadly applied to any information processing system that generates code from visual designs, such as generating front-end code (HTML / CSS, etc.) for websites, building UI screens for mobile applications, or automatic generation tools for forms and reports.

[0079] Furthermore, while VLM210 and LLM220 were logically described separately as AI models to be used, the entire system may be configured using a single multimodal platform model that combines integrated processing of both text and images with advanced logical reasoning capabilities. In addition, the output of evaluation results by the evaluation unit 126 and the correction approach by the correction unit 128 may also be independent of LLM and, alternatively, configured so that a dedicated algorithm or rule-based engine parses the output structured data (error report) and corrects the code. [Explanation of symbols]

[0080] 100 Information Processing Devices 110 First Information Processing Unit 112, 122 Acquisition Department 114 1st generation part 116 Second generation part 120 Second Information Processing Unit 124 Execution Unit 126 Evaluation Department 128 Correction section 130 Storage section 132 Sample Data 134 Various prompts 136 Hardcode 138 Template Functions 140 Configuration Data 200 AI servers 210 Visual Language Models 220 language models

Claims

1. An acquisition unit that acquires sample data showing the design layout, A first generation unit generates a first code that describes the arrangement and style of the elements included in the design using fixed values, based on the aforementioned sample data. The system includes a second generation unit that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​contained in the first code into variables or control structures, The first code is executable code that does not include variable definitions or logical iteration, but describes the arrangement and style of the elements using absolute values. Information processing device.

2. An acquisition unit that acquires sample data showing the design layout, A first generation unit generates a first code that describes the arrangement and style of the elements included in the design using fixed values, based on the aforementioned sample data. The system includes a second generation unit that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​contained in the first code into variables or control structures, The second generation unit extracts constants related to the style from the first code and further generates setting data that defines the constants. The second generation unit generates the second code and the setting data separately. Information processing device.

3. The second generation unit identifies text strings or image reference information contained in the first code and replaces them with dynamic variables determined according to data input from an external source. The information processing apparatus according to claim 1.

4. The second generation unit identifies a sequence of commands that are repeated in a predetermined pattern in the first code, and converts the sequence into an iterative control structure. The information processing apparatus according to claim 1.

5. An acquisition unit that acquires sample data showing the design layout, A first generation unit generates a first code that describes the arrangement and style of the elements included in the design using fixed values, based on the aforementioned sample data. The system includes a second generation unit that analyzes the first code using a language model and generates a generalized second code by converting at least a portion of the fixed values ​​contained in the first code into variables or control structures, The aforementioned sample data includes structured data that defines the structure of the design, The first generation unit generates the first code by mapping the elements included in the structured data to instructions of the target application programming interface. Information processing device.

6. The aforementioned sample data includes an image of the aforementioned design, The first generation unit generates the first code by visually analyzing the image using a visual language model. The information processing apparatus according to claim 1.

7. The second generation unit inputs at least one of the image of the sample data or structured data obtained from the sample data as additional information to the language model, in addition to the first code, and generates the second code. The information processing apparatus according to claim 1.

8. An evaluation unit evaluates the style of the candidate image by comparing the candidate image generated using the second code with the sample data image, The system further includes a modification unit that modifies the setting data based on the evaluation results from the evaluation unit. The information processing apparatus according to claim 2.

9. Computers Obtain sample data showing the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. A generalized second code is generated by analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures. The first code is executable code that does not include variable definitions or logical iteration, but describes the arrangement and style of the elements using absolute values. Information processing methods.

10. On the computer, Obtain sample data that shows the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. By analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures, a generalized second code is generated. The first code is executable code that does not include variable definitions or logical iteration, but describes the arrangement and style of the elements using absolute values. program.

11. Computers Obtain sample data showing the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. A generalized second code is generated by analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures. The constants related to the style are extracted from the first code, and setting data defining the constants is further generated. The second code and the setting data are generated separately. Information processing methods.

12. On the computer, Obtain sample data that shows the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. By analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures, a generalized second code is generated. The constants related to the style are extracted from the first code, and setting data defining the constants is further generated. The second code and the setting data are generated separately. program.

13. Computers Obtain sample data showing the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. A generalized second code is generated by analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures. The aforementioned sample data includes structured data that defines the structure of the design, The first code is generated by mapping the elements contained in the structured data to the instructions of the target application programming interface. Information processing methods.

14. On the computer, Obtain sample data that shows the design layout, Based on the aforementioned sample data, a first code is generated that describes the arrangement and style of the elements included in the design using fixed values. By analyzing the first code using a language model and converting at least some of the fixed values ​​contained in the first code into variables or control structures, a generalized second code is generated. The aforementioned sample data includes structured data that defines the structure of the design, The first code is generated by mapping the elements contained in the structured data to the instructions of the target application programming interface. program.

Images