Intelligent WinForm interface generation method and system based on state collaboration and multi-mode perception

Abstract

The invention relates to the technical field of software engineering automation driven by artificial intelligence, in particular to a WinForm interface intelligent generation method and system based on state collaboration and multi-mode perception. According to the method, a directed graph G = (V, E) and a uniform state sequence sigma are constructed, and a code generation task is formalized into workflows for cooperation among multiple agents and between people and the agents; the agent defines an execution task; according to the pointing sequence of the edges, the first agent node is a demand analysis agent and is used for extracting a service table name set and an interface type; the interface type is divided into a single-table interface and a multi-table interface; the end agent is a code generation agent and is used for outputting a code for defining a CRUD interface; the state sequence promotes interaction and feedback between the person and the intelligent agent; specifically, a user demand text is processed by a demand analysis agent, and then execution and retry of each agent are guided in combination with current state data and an edge set V until a code generation agent outputs a code. On the basis of man-machine state collaboration and a multi-mode UI sensing mechanism, end-to-end intelligent generation from multi-mode UI design to high-quality codes is achieved, and the code generation efficiency and the automation degree are remarkably improved while it is ensured that a design draft is precisely restored.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence driven software engineering automation, and in particular to a WinForm interface intelligent generation method and system based on state cooperation and multi-modal perception. BACKGROUND

[0002] Under the WinForm framework, traditional GUI (Graphical User Interface) development relies on manual dragging of controls and hard-coded layout, and there are significant defects in the face of multi-table and multi-field scenarios: manual layout is inefficient and has high maintenance costs, especially when the interface changes on a large scale, it is difficult to automatically adjust. Although the existing code generator can partially automate the CRUD (Create, Read, Update and Delete) interface generation, it has double limitations: on the one hand, it relies on pre-defined structured text layout rules (such as CN117435271A requires pre-defined control hierarchy and coordinate information), which leads to the need for manual conversion of UI design diagrams into natural language descriptions in actual development, resulting in high conversion error rates and inability to adapt to dynamic needs; on the other hand, the application of large model multi-modal capabilities is limited, such as in CN119987734A, although a large model is used to generate a use case diagram, the form of the use case diagram is actually a PlantUML language description, which is still in text form, but UI images are often the information form delivered by product personnel to technical personnel in actual production, so how to accurately identify the position of image information is the most core problem. In addition, in the code generation task, because Winform interface needs to declare a large amount of control information (such as control declaration, instantiation, property setting, etc.), with the increase of business table fields and the demand for multi-table association interface generation, the amount of code will far exceed the general limit of the output length of the large model. The current technical solution (CN119987734A, CN117435271A) innovatively proposes to use a large language model to generate interface code, however, the multi-field and multi-table code generation task cannot completely rely on the generation of the large model, and the code generation will be incomplete and resources will be wasted beyond the output capacity limit of the large model, and the illusion problem of the large model generation cannot be ignored, especially in the strict problem of code generation. More fundamentally, the existing technical solutions (including existing multi-agent cooperation frameworks) focus more on the autonomous cooperation between agents, and the process is often pre-defined and one-way, the intermediate process is a "black box" to human users, lacking effective intervention and guidance interfaces. This leads to its inability to dynamically adjust the code generation process according to real-time task status, user feedback and intermediate products, and the degree of intelligence is limited, and it is difficult to form an end-to-end closed loop that truly meets the user's expectations.

[0003] In summary, the existing Winform table management interface generation technology has the following bottlenecks:

[0004] 1. Manual layout relies on dragging and dropping, which is extremely inefficient and has high maintenance costs when requirements change. Existing automation solutions usually use strategies such as "loop + fixed offset" or "text layout description" to generate interfaces, resulting in rigid layouts that are difficult to adapt to complex business scenarios.

[0005] 2. Insufficient utilization of multimodal UI diagram information; for example, although CN117435271A relies on accurate interface layout descriptions to generate complete interface code, the UI design diagrams need to be manually converted into text-based layout descriptions or coordinate values ​​beforehand, resulting in a high conversion error rate and high labor costs; similarly, although CN119987734A uses large models to generate use case diagrams, the use case diagrams are still in the form of text-based PlantUML language descriptions. This does not conform to conventional UI delivery methods.

[0006] 3. The output length limit of large language models cannot support the full code generation (declaration, instantiation, property setting) of massive controls in multi-field, multi-table CRUD interfaces. If the output length limit of large models is exceeded, the generated code will be incomplete. Neither CN117435271A nor CN119987734A proposes a further solution.

[0007] 4. Code generation tasks differ from general text generation, demanding extremely high accuracy and logical consistency. Relying entirely on large models for end-to-end generation can easily introduce semantic errors, logical flaws, or security vulnerabilities. Furthermore, while existing multi-agent collaborative frameworks break down code generation tasks into multiple independent agents, improving the professionalism and accuracy of each process to some extent, the intermediate processes remain a "black box" for the user. Humans cannot participate in decision-making, potentially leading to a significant discrepancy between the final output of the code generation task and the user's expectations. Summary of the Invention

[0008] To overcome the shortcomings of existing management interface generation technologies, this invention proposes a WinForm interface intelligent generation method and system based on state collaboration and multimodal awareness, which solves the problem of dynamic intelligent conversion between multimodal UI design drawings and actual interface code, as well as the problem of user-controllable participation in the conversion process.

[0009] This invention proposes a WinForm interface intelligent generation method based on state collaboration and multimodal perception. It constructs an intelligent agent and a directed graph G=(V,E), and formalizes the code generation task into a multi-agent collaborative workflow. V is the set of intelligent agent nodes; E is the set of edges. The intelligent agents are defined with execution tasks, and the edges in the edge set are used to guide the execution order and data flow of the intelligent agent nodes.

[0010] Based on the order of the edges, the first agent node is the requirements analysis agent, which is used to extract the set of business table names and interface types; the interface types are divided into single-table interfaces and multi-table interfaces; the last agent is the code generation agent, which is used to output the code that defines the CRUD interface.

[0011] The user requirement text is processed by the requirement analysis agent, and then the data is transmitted between the agents according to the edge set V, and processed in sequence until the code generation agent outputs the code.

[0012] Preferably, the agent node further includes: a database query agent, a prototype agent, a location code agent, a hard-coded agent, and a code generation agent; the method includes the following steps:

[0013] Construct the intelligent agent and the directed graph G = (V, E);

[0014] The demand analysis agent accepts and parses the user's natural language requirements, and outputs structured data containing business table names and interface types; for multi-table interfaces, the user can specify the associated fields of each business table through interface interaction.

[0015] The database query agent connects to the database based on structured data to perform table structure queries and obtain the table structure information of the business tables. The table structure information includes: the name and data type of all fields.

[0016] Prototype information is obtained through the prototype agent; the prototype information is the parsing result of the prototype uploaded by the user, or the interface layout description of the automatically generated prototype.

[0017] The base code is generated by hard-coding the intelligent agent, including control declaration, instantiation, and property setting;

[0018] The location code agent generates key location codes based on prototype diagram information;

[0019] The code-generated agent combines base code and key location code to generate interface code for defining the graphical interface.

[0020] Preferably, when the user has a prototype image, the prototype image agent calls the OCR interface to parse the prototype image uploaded by the user and generates structured data with precise location information. The location code agent generates key location codes based on this structured data. When the user does not have a prototype image, the prototype image agent searches for historical similar table names, obtains the layout information of similar business table interfaces, and generates an interface layout description. The location code agent generates key location codes based on this interface layout description.

[0021] Preferably, when the user has no prototype image, the prototype image agent uses similarity to retrieve the UI design descriptions of historically related tables based on auxiliary information, and uses a few-shot hint engineering strategy to use the UI design descriptions of historically related similar tables as reference examples for the prototype image agent to automatically generate the UI design of each business table; the auxiliary information includes the business table name.

[0022] Preferably, a state center is also provided for unified management of structured state data for collaboration among multiple agents and between humans and agents;

[0023] State data is defined as: ∑={∑0,∑1,∑2,…,∑ t ,…,∑ T}; T is the total number of states, 1≤t≤T, ∑ t For the t-th state data, each state data ∑ t It contains one or more pieces of information, each of which includes a name and content; ∑0 represents the initial state, which includes user requirement information and workflow exception information;

[0024] The state center adds state data ∑1 based on the return results of the user demand agent, which includes information from the initial state and the return results of the demand analysis agent.

[0025] The status center pushes status data ∑1 to the user to confirm whether the requirement has been correctly parsed; if the user gives positive feedback, the status center adds status data ∑2, which contains the final requirement information; if the user gives negative feedback, the requirement analysis agent executes again, and the execution result updates status data ∑1.

[0026] After the database query agent completes the database query, the status center adds status data ∑3, which contains the table structure information of all business tables output by the database query agent; if the status data ∑2 is a multi-table task, the status center will push status data ∑3 to the user to select the associated fields.

[0027] Based on the state data ∑3, the state center adds the associated fields of each business table. After obtaining the state data ∑4, the hard-coded agent generates the interface base code of the business table corresponding to the user requirements. The state center also adds state data ∑5 containing the declaration, instantiation and attribute settings of controls. The state center pushes a message to the user about the prototype diagram generation method and generates state data ∑6 containing the prototype diagram generation method, state data ∑7 containing prototype diagram information and state data ∑8 containing key position code based on user feedback.

[0028] Preferably, when using a prototype image uploaded by the user, the prototype image information in ∑7 is the prototype image OCR parsing result; when using a prototype image automatically generated by the prototype image agent, the prototype image information in ∑7 is the interface layout description of the prototype image.

[0029] Preferably, after the status data ∑7 is added, the status center confirms with the user whether the prototype diagram is correct;

[0030] If the user confirms an error, the prototype diagram agent will execute the corresponding prototype diagram generation logic again based on the prototype diagram generation method confirmed by state data ∑6, and the execution result will be updated to state data ∑7.

[0031] Once the user confirms that the code is correct, the location code agent generates key location code statements in the WinForm interface based on the prototype diagram information in the state data ∑7 and the instruction constraints containing business rules. The business rules are the unified naming rules for controls. The state center adds state data ∑8, which contains key location code statements.

[0032] After the state data ∑8 is added, the state center asks the user to confirm whether to start generating code. If the user confirms, the code generation agent retrieves the state data ∑5, which represents the hard-coded code, and the state data ∑8, which represents the key position code, from the state center. Based on the state data ∑5, and according to the unique identifier of the control name, it replaces the position code of all controls in the state data ∑5 with the position code statement of that control in the state data ∑8, thus completing the code generation.

[0033] This invention proposes a WinForm interface intelligent generation system based on state coordination and multimodal awareness, comprising:

[0034] A requirements analysis agent is used to extract the set of business table names and interface types.

[0035] The database query agent parses the business table names based on the results returned by the requirements analysis agent and connects to the database to query the table structure. The table structure information includes: the names and data types of all fields, and the field names include Chinese names and / or English names.

[0036] Hard-coded agents use a fixed offset strategy for control positions to generate base code;

[0037] The prototype agent is used to parse the interface prototype diagram; the interface prototype diagram is the interface design diagram uploaded by the user; or the interface design information automatically generated based on the table structure information output by the database agent and the interface layout description of the retrieved historical similar tables.

[0038] The location code agent generates location codes containing key positions in the interface prototype based on the prototype diagram parsing results; or generates location codes containing key positions of interface controls based on automatically generated interface layout information.

[0039] The code-generating agent combines positional code and base code to generate the final interface code.

[0040] Preferably, the database agent uses an API interface to obtain the database structure of the business tables.

[0041] The present invention proposes a storage medium storing a computer program, which, when executed, is used to implement the WinForm interface intelligent generation method based on state collaboration and multimodal perception.

[0042] The advantages of this invention are:

[0043] (1) This invention defines a multi-agent collaborative process, transforming disordered tool calls, API calls, database interactions, etc., into ordered, dynamic, and adaptive agent collaboration, greatly improving the robustness and flexibility of the system. By defining dedicated agents for each stage, it is possible to deeply understand the underlying needs of tasks at each stage and transform them into structured tasks that can be executed by machines. No strict parameter configuration is required, and it supports natural language requirement understanding, achieving a high degree of intelligence and understanding capabilities.

[0044] (2) This invention makes full use of UI design information. On the one hand, it uses OCR technology to accurately identify the absolute position coordinates of controls in the design; on the other hand, it uses the logical reasoning ability of the large language model to deeply understand the relative positional relationship between controls through the absolute positional information, and generates position code that retains only the relative positional relationship between controls, thereby realizing the intelligent conversion from design to interface code, which not only greatly improves the accuracy of interface generation, but also greatly reduces the manual conversion cost.

[0045] (3) This invention introduces a state center as the coordination core, which not only efficiently and structurally transmits key information (in JSON format) among various agents, but more importantly, allows users to interact with each agent through the state center. The state center deeply integrates user decisions and feedback into the code generation process, realizing closed-loop iteration and continuous optimization. This makes code generation not a one-time process, but an iterative process that is continuously optimized, controllable, transparent, and constantly approaches the user's final needs, greatly enhancing the accuracy and controllability of interface code generation.

[0046] (4) Under the multi-agent collaboration framework, this invention uses hard-coded agents to generate base code and location code agents to generate key location code, and is supplemented by a dynamic coordinate injection mechanism. This innovatively breaks through the limitations of completeness and accuracy of code generation that rely entirely on large models in multi-field and multi-table scenarios, as well as the limitation that traditional hard coding cannot dynamically capture UI design. This greatly improves the accuracy, controllability and generalization ability of the Winform multi-field and multi-table management interface code generation task. Attached Figure Description

[0047] Figure 1 This is a flowchart of a WinForm interface intelligent generation method based on state collaboration and multimodal perception proposed in this invention;

[0048] Figure 2 This is a flowchart illustrating the collaborative process of various agents in a WinForm interface intelligent generation method based on state collaboration and multimodal perception proposed in this invention.

[0049] Figure 3 This is a diagram illustrating the generation effect of a hard-coded intelligent agent in a WinForm interface intelligent generation method based on state collaboration and multimodal perception proposed in this invention.

[0050] Figure 4 This is a user-uploaded UI design drawing in the WinForm interface intelligent generation method based on state collaboration and multimodal perception proposed in this invention.

[0051] Figure 5 This is a diagram showing the final generated intelligent agent in the WinForm interface intelligent generation method based on state collaboration and multimodal perception proposed in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 , Figure 2 As shown, the present invention proposes a WinForm interface intelligent generation method based on state collaboration and multimodal perception, and the specific steps are as follows.

[0054] Step 1: Construct a multi-agent collaborative workflow. This method first formalizes the complex code generation task into a predefined, executable collaborative workflow. This workflow is constructed as a directed acyclic graph (DAG), denoted as G, G = (V, E). Here, V is the set of nodes in the graph, where each node v ∈ V represents an independent, functionally specialized agent or a decision point. The agent nodes are responsible for executing specific tasks, while the decision nodes do not execute tasks but dynamically select the next execution path based on specific conditions of the current state. E is the set of edges in the graph, representing the execution order and data flow between nodes, with each edge e = (v...). i ,v j )∈E means at node v i Once execution is complete, control and the updated state will be passed to node v. j Edge types include sequential edges and conditional edges. Sequential edges define a fixed execution sequence, while conditional edges refer to dynamic routing of the workflow starting from a decision node and based on its evaluation result.

[0055] Step 2: Initialize the unified state center chain. Users interact with each agent through the state center. The state center chain is denoted as ∑={∑0,∑1,...,∑ t ,...,∑ T}, where ∑ t This represents the data at the t-th state during task execution, and is recorded as a set of key-value pairs, such as:

[0056]

[0057] in, This represents the key and value of a certain piece of information recorded in the state center at state t.

[0058] At the start of each task execution, the state center is initialized, generating an initial state ∑0. The initial state contains at least the original information submitted by the user:

[0059] ∑0={"user_request":"Please help me generate xxx","error":null}

[0060] Where: user_request is the natural language request text entered by the user; error is used to capture and log any exceptions during workflow execution.

[0061] Step 3: User Input Processing. The requirement analysis agent receives the user's natural language requirement R, such as: "Help me generate a management interface for the User and Department tables." The requirement analysis agent uses a large language model to perform semantic parsing of the user's requirement according to specific instructions, and finally outputs the following structured data:

[0062]

[0063] Where, T={t1,t2,...,t m ,...,t M} represents the set of M business table names parsed from user requirements, t m O1 represents the name of the m-th business table; τ represents the interface type, which is an enumeration value ("single table" or "multiple tables"). P1 represents the instruction template followed by the requirements analysis agent. O1 is {(User, Department), "multiple tables"}, and P1 is:

[0064] ```

[0065] You are a table name resolution expert. Your task is to accurately extract table names from user requirements.

[0066] rule:

[0067] 1. Carefully analyze user input.

[0068] 2. Identify the table names. Table names usually begin with a capital letter and contain letters, numbers, and underscores.

[0069] 3. Strictly follow the format to return: [TABLE_NAME]: table name

[0070] 4. If the table name cannot be determined, return: [TABLE_NAME]: UNKNOWN

[0071] Example:

[0072] - The user said, "Please help me generate the WinForms interface code for the user table" → [TABLE_NAME]: User table

[0073] - The user mentioned "the interface for generating the user information table" → [TABLE_NAME]: User Information Table

[0074] - The user said "Design the interface for the ProductInfo table" → [TABLE_NAME]:ProductInfo

[0075] - The user mentioned "WinForms interface for the CT_Test table" → [TABLE_NAME]:CT_Test

[0076] ```

[0077] Step 3: User Requirement Confirmation. After completing the parsing of business table names and interface types in Step 1, the latest status data ∑1 is appended to the status center. ∑1 includes all business table names and interface type information involved in this requirement. The status center pushes the status data ∑1 to the user for confirmation that the requirement has been parsed correctly.

[0078] ∑1={"user_request":R||P1,"llm_response":O1,"error":null}

[0079] Where: user_request is the natural language request text and prompt words entered by the user; llm_response is the result O1 returned by the requirement analysis agent; error is used to capture and log any exceptions during workflow execution.

[0080] In step 3, if the user's requirements are incorrect, step 2 is re-executed to update the status data ∑1; if the user confirms that the requirements are correct, status data ∑2 is added to the status center, which contains the final requirement information, and then step 4 is executed.

[0081] ∑2={"type":"single table or multiple tables","tableset":[table1,table2,…],"error":null}

[0082] Here, `type` indicates the type of interface generation task, which can be a single table or multiple tables. `tableset` is the collection of business tables involved in the requirement; `error` is used to capture and log any exceptions during workflow execution.

[0083] Step 4: Database Query. The database query agent parses the business table names from the latest status data ∑2 in the status center and automatically performs database connection and table structure query by calling the unified API interface, finally returning the table structure information O2 as shown below; and the status center adds new status data ∑3, which contains the table structure information of all business tables output by the database query agent.

[0084] O2 = API GetSchema (T,τ)={Ω1,Ω2,...,Ω m ,...,Ω M} (3)

[0085] Among them, Ω mThis represents the table structure information of the m-th business table mentioned in the user requirements, including the Chinese and English names and data types of all fields in the table. The data is in JSON format, with the table name as the key and a list of basic information for all fields in the table as the value. The basic information for each field includes its Chinese and English name and its data type. The detailed format is as follows:

[0086]

[0087] ∑3={"table_info":["table1":"xxx","table2":"xxx",..],"error":null}

[0088] `table_info` represents the set of table structure information for all business tables in this task. The table structure information for `table1` is `xxx`, the table structure information for `table2` is `xxx`, and so on. The table structure information includes the Chinese name, design name, type, and data dictionary value of all fields, as shown in the table below.

[0089]

[0090] Step 5: Selecting Join Fields for Multiple Tables. Based on status data ∑2, decide whether to send a message to the user to select join fields: If status data ∑2 represents a single table, no join fields are needed, and proceed directly to Step 6; if status data ∑2 represents multiple tables, the status center needs to pop up a message and wait for the user to select join fields for each table. After the selection is completed, add the join fields for each business table to status data ∑3 to obtain status data ∑4; then proceed to Step 6.

[0091] ∑4={"table_info":["table1":"xxx","table2":"xxx",..],

[0092] "foreignkey":["table1":"ID","table2":"InnerID",..],

[0093] "error":null}

[0094] Here, `foreignkey` represents the set of related fields for all business tables in this task. For example, the related field for `table1` is `ID`, and the table structure information for `table2` includes `InnerID`, etc. `table_info` represents the set of table structure information for all business tables in this task.

[0095] Step 6: The hard-coded agent uses a strategy of "looping controls + fixed offset" to generate the interface base code for the table as required. After generation, a new state data ∑5 is added to the state center, which includes control declaration, instantiation, and attribute settings; the attribute settings include name (table name), size (size), text (text content), and location (location information).

[0096] ∑5={"hard_code":"xxx","error":null}

[0097] Here, hard_code represents the base code of the table management interface.

[0098] The hard-coded agent uses a "loop control + fixed offset" strategy to generate the interface base code for the table specified in the requirements. The method is as follows: loop through each field in the business table, generate a label control and a text box control for each field, and generate WinForm interface code for each control (specifically including control declaration, instantiation, and property setting (name, size, text, location, etc.)). The fixed offset means that when setting the location property of each control, by default, the position of the next control in the loop is consistent with the position of the current control in the X coordinate, while the Y coordinate is fixedly offset. This forms... Figure 3 , Figure 4 , Figure 5 The interface layout shown.

[0099] Step 7: Confirm Prototype Generation Method. After adding the latest state data ∑5 to the state center, the state center pushes a message to the user regarding the prototype generation method. After the user provides feedback, the state center adds state data ∑6 containing the prototype generation method information. The prototype agent obtains the prototype and its information based on the generation method information (manual upload or automatic generation) in state data ∑6.

[0100] ∑6={"UIupload_type":"Manual Upload","error":null}

[0101] The UIupload_type parameter indicates the method of uploading the prototype image, with a value of "manual upload" or "automatic generation".

[0102] Specifically, prototype generation can be divided into the following two cases based on the generation method:

[0103] Step 7.1: When the user selects "Manual Upload," the prototype agent parses the UI design draft uploaded by the user. The prototype agent automatically calls the unified OCR interface for image recognition, identifying element names, positions, etc., in the design draft, and returning structured information with precise absolute position information, denoted as O3.

[0104]

[0105] For each image uploaded by the user, the prototype agent calls a unified OCR interface in parallel to generate JSON-formatted location information for each UI design image, denoted as . The detailed format is as follows:

[0106]

[0107] Step 7.2: When the user selects "Auto Generate", the prototype agent generates the prototype diagram based on the table structure information Ω. m Intelligent UI design generation. The prototype agent utilizes a large language model to complete the UI design of specific business tables by following specific instructions, ultimately outputting a UI layout description. The initial generation can retrieve UI design descriptions of historically related tables based on similarity using auxiliary information such as table names, and employs a few-shot hinting engineering strategy to use these descriptions as reference examples for the agent's UI design. Furthermore, the agent can receive user feedback and repeatedly adjust the UI design until the user confirms the prototype is correct.

[0108] Step 8: Based on the prototype information, the State Center adds state data ∑7, which contains prototype information, namely the OCR parsing result of the UI design uploaded by the user or the automatically generated natural language description of the UI design. In this step, the State Center confirms with the user whether the prototype information in state data ∑7 is correct; if incorrect, it re-obtains the prototype information according to the prototype generation method confirmed in state data ∑6; if correct, it performs subsequent operations to obtain the key location code based on the prototype information.

[0109] ∑7={"UI_description":"xxx","error":null}

[0110] Here, UI_description represents the description of the prototype diagram.

[0111] When the prototype image is uploaded manually, the UI_description is structured data that includes precise location descriptions for different areas (such as date, project name, processing status, and data identifier). These precise location descriptions can be expressed at the pixel level. For example, the structured data code for "approval date" is as follows:

[0112]

[0113]

[0114] When the prototype image is uploaded using the "Auto-Generate" method, the UI_description is as follows:

[0115] "The following is a WinForm data entry and query page designed for table xxx. The control layout is clear, and the fields are organized using multi-column grouping and row-based arrangement. The following is a detailed analysis of the interface control layout:..."

[0116] In practice, a detailed analysis of the layout of interface controls can be performed by describing the spatial positions of controls from the perspectives of overall layout structure, row-by-row analysis, row and column layout structure, and summary of control types, thereby obtaining a description of the interface layout.

[0117] For example, the overall layout structure is described as follows:

[0118] "The entire interface adopts a 3-column layout (left, middle, and right), with each column containing several rows."

[0119] Each row consists of a label and an input control (such as a TextBox, ComboBox, etc.).

[0120] All controls are left-aligned, with labels on the left and input fields on the right.

[0121] The spacing between controls is uniform, conforming to standard WinForm layout specifications.

[0122] Step 9: The location code agent generates key location codes based on the prototype diagram information; the state center adds state data ∑8 containing key location code statements; specifically, the location code agent uses the large language model to generate key location code statements for all controls in the Winform interface based on the state data ∑7.

[0123] ∑8={"location_code":"xxx","error":null}

[0124] Here, `location_code` represents the location code generated from the prototype diagram for all controls in the table management interface, as shown in the example below:

[0125] “

[0126] lbl_input_CNMBondLending_ID=new System.Drawing.Point(10,20);

[0127] txt_input_CNMBondLending_ID=new System.Drawing.Point(100,20);

[0128] lbl_input_CNMBondLending_DeclareDate=new System.Drawing.Point(290,20);

[0129] txt_input_CNMBondLending_DeclareDate=new System.Drawing.Point(380,20);

[0130] lbl_input_CNMBondLending_Symbol=new System.Drawing.Point(10,60);

[0131] txt_input_CNMBondLending_Symbol=new System.Drawing.Point(100,60);

[0132] lbl_input_CNMBondLending_TDate=new System.Drawing.Point(290,60);

[0133] txt_input_CNMBondLending_TDate=new System.Drawing.Point(380,60);

[0134] lbl_input_CNMBondLending_OpenPrice=new System.Drawing.Point(10,100);

[0135] txt_input_CNMBondLending_OpenPrice=new System.Drawing.Point(100,100);

[0136] lbl_input_CNMBondLending_ClosedPrice=new System.Drawing.Point(290,100);

[0137] txt_input_CNMBondLending_ClosedPrice=new System.Drawing.Point(380,100);

[0138] lbl_input_CNMBondLending_HighPrice=new System.Drawing.Point(10,140);

[0139] txt_input_CNMBondLending_HighPrice=new System.Drawing.Point(100,140);

[0140] lbl_input_CNMBondLending_LowPrice=new System.Drawing.Point(290,140);

[0141] txt_input_CNMBondLending_LowPrice=new System.Drawing.Point(380,140);

[0142] lbl_input_CNMBondLending_AvgPrice=new System.Drawing.Point(10,180);

[0143] txt_input_CNMBondLending_AvgPrice=new System.Drawing.Point(100,180);

[0144] lbl_input_CNMBondLending_VoTurnover=new System.Drawing.Point(290,180);

[0145] txt_input_CNMBondLending_VoTurnover=new System.Drawing.Point(380,180);

[0146] lbl_input_CNMBondLending_Memo=new System.Drawing.Point(10,220);

[0147] txt_input_CNMBondLending_Memo=new System.Drawing.Point(100,220);

[0148] btn_input_CNMBondLending=new System.Drawing.Point(150,280);

[0149] btn_cancel_CNMBondLending=new System.Drawing.Point(250,280);

[0150] "

[0151] Special Note: When using OCR to recognize control positions, it relies on the size of the prototype image and uses the top left corner of the uploaded image as the coordinate origin. However, manual screenshots and the actual interface often have errors, causing the OCR results to be unsuitable for direct use as control positions in the code. Therefore, the prompts constrain the large model to understand the relative positional relationships between controls, recalibrate the coordinate origin, and generate key position code that retains only the relative positional relationships between controls and conforms to the coordinate system of the actual interface.

[0152] Step 10: The State Center confirms with the user whether to start generating code. If the user confirms, the code generation agent obtains state data ∑5, representing hard-coded code, and state data ∑8, representing key location code, from the State Center. Based on state data ∑5, according to the unique identifier of the control name, the position code of all controls in state data ∑5 is replaced with the key location code statement of that control in state data ∑8. The remaining control declarations, instantiations, and attribute settings (except for Location) remain unchanged, thus completing code generation.

[0153] Of course, those skilled in the art will recognize that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0154] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0155] The technologies, shapes, and structures not described in detail in this invention are all known technologies.

Claims

1. A method for intelligently generating WinForm interfaces based on state collaboration and multimodal awareness, characterized in that, Construct intelligent agents and a directed graph G = (V, E), formalizing the code generation task as a multi-agent collaborative workflow; V is the set of intelligent agent nodes; E is the set of edges; intelligent agents are defined to perform tasks, and the edges in the edge set are used to guide the execution order and data flow of intelligent agent nodes; Based on the order of the edges, the first agent node is the requirements analysis agent, which is used to extract the set of business table names and interface types; the interface types are divided into single-table interfaces and multi-table interfaces; the last agent is the code generation agent, which is used to output the code that defines the CRUD interface. The user requirement text is processed by the requirement analysis agent, and then the data is transmitted between the agents according to the edge set V, and processed in sequence until the code generation agent outputs the code.

2. The WinForm interface intelligent generation method based on state collaboration and multimodal perception as described in claim 1, characterized in that, The agent node also includes: a database query agent, a prototype agent, a location code agent, a hard-coded agent, and a code generation agent; the method includes the following steps: Construct the intelligent agent and the directed graph G = (V, E); The demand analysis agent accepts and parses the user's natural language requirements, and outputs structured data containing business table names and interface types; for multi-table interfaces, the user can specify the associated fields of each business table through interface interaction. The database query agent connects to the database based on structured data to perform table structure queries and obtain the table structure information of the business tables. The table structure information includes: the name and data type of all fields. Prototype information is obtained through the prototype agent; the prototype information is the parsing result of the prototype uploaded by the user, or the interface layout description of the automatically generated prototype. The base code is generated by hard-coding the intelligent agent, including control declaration, instantiation, and property setting; The location code agent generates key location codes based on prototype diagram information; The code-generated agent combines base code and key location code to generate interface code, which is used to define the graphical interface.

3. The WinForm interface intelligent generation method based on state collaboration and multimodal perception as described in claim 2, characterized in that, When a user has a prototype image, the prototype image agent calls the OCR interface to parse the prototype image uploaded by the user and generate structured data with precise location information. The location code agent then generates key location codes based on this structured data. When the user has no prototype image, the prototype image agent retrieves historical similar table names, obtains the layout information of similar business table interfaces, and generates an interface layout description. The location code agent generates key location codes based on this interface layout description.

4. The WinForm interface intelligent generation method based on state collaboration and multimodal perception as described in claim 3, characterized in that, When the user has no prototype image, the prototype image agent uses auxiliary information to retrieve the UI design descriptions of historically related tables based on similarity. It also uses a few-shot hint engineering strategy to use the UI design descriptions of historically related similar tables as reference examples for the prototype image agent to design the UI, so as to automatically generate the UI design of each business table. The auxiliary information includes the name of the business table.

5. The WinForm interface intelligent generation method based on state collaboration and multimodal awareness as described in claim 3, characterized in that, A state center is also set up to uniformly manage the structured state data of collaboration between multiple agents and between humans and agents; State data is defined as: ∑={∑0,∑1,∑2,…,∑ t ,…,∑ T }; T is the total number of states, 1≤t≤T, ∑ t For the t-th state data, each state data ∑ t It contains one or more pieces of information, each of which includes a name and content; ∑0 represents the initial state, which includes user requirement information and workflow exception information; The state center adds state data ∑1 based on the return results of the user demand agent, which includes information from the initial state and the return results of the demand analysis agent. The status center pushes status data ∑1 to the user to confirm whether the requirement has been correctly parsed; if the user gives positive feedback, the status center adds status data ∑2, which contains the final requirement information; if the user gives negative feedback, the requirement analysis agent executes again, and the execution result updates status data ∑1. After the database query agent completes the database query, the status center adds status data ∑3, which contains the table structure information of all business tables output by the database query agent; if the status data ∑2 is a multi-table task, the status center will push status data ∑3 to the user to select the associated fields. Based on the state data ∑3, the state center adds the associated fields of each business table. After obtaining the state data ∑4, the hard-coded agent generates the interface base code of the business table corresponding to the user requirements. The state center also adds state data ∑5 containing the declaration, instantiation and attribute settings of controls. The state center pushes a message to the user about the prototype diagram generation method and generates state data ∑6 containing the prototype diagram generation method, state data ∑7 containing prototype diagram information and state data ∑8 containing key position code based on user feedback.

6. The WinForm interface intelligent generation method based on state collaboration and multimodal awareness as described in claim 5, characterized in that, When using user-uploaded prototype images, the prototype image information in ∑7 is the prototype image OCR parsing result; when using prototype images automatically generated by the prototype image agent, the prototype image information in ∑7 is the interface layout description of the prototype image.

7. The WinForm interface intelligent generation method based on state collaboration and multimodal awareness as described in claim 1, characterized in that, After the status data ∑7 is added, the status center confirms with the user whether the prototype diagram is correct; If the user confirms an error, the prototype diagram agent will execute the corresponding prototype diagram generation logic again based on the prototype diagram generation method confirmed by state data ∑6, and the execution result will be updated to state data ∑7. Once the user confirms that the code is correct, the location code agent generates key location code statements in the WinForm interface based on the prototype diagram information in the state data ∑7 and the instruction constraints containing business rules. The business rules are the unified naming rules for controls. The state center adds state data ∑8, which contains key location code statements. After the state data ∑8 is added, the state center asks the user to confirm whether to start generating code. If the user confirms, the code generation agent retrieves the state data ∑5, which represents the hard-coded code, and the state data ∑8, which represents the key position code, from the state center. Based on the state data ∑5, and according to the unique identifier of the control name, it replaces the position code of all controls in the state data ∑5 with the position code statement of that control in the state data ∑8, thus completing the code generation.

8. A WinForm interface intelligent generation system based on state collaboration and multimodal perception, characterized in that, include: A requirements analysis agent is used to extract the set of business table names and interface types. The database query agent parses the business table names based on the results returned by the requirements analysis agent and connects to the database to query the table structure. The table structure information includes: the names and data types of all fields, and the field names include Chinese names and / or English names. Hard-coded agents use a fixed offset strategy for control positions to generate base code; The prototype agent is used to parse the interface prototype diagram; the interface prototype diagram is the interface design diagram uploaded by the user; or the interface design information automatically generated based on the table structure information output by the database agent and the interface layout description of the retrieved historical similar tables. The location code agent generates location codes containing key positions in the interface prototype based on the prototype diagram parsing results; or generates location codes containing key positions of interface controls based on automatically generated interface layout information. The code-generating agent combines positional code and base code to generate the final interface code.

9. The WinForm interface intelligent generation system based on state collaboration and multimodal perception as described in claim 8, characterized in that, The database agent uses an API interface to obtain the database structure of the business tables.

10. A storage medium, characterized in that, The system contains a computer program that, when executed, implements the WinForm interface intelligent generation method based on state collaboration and multimodal awareness as described in any one of claims 1-7.

Images