System for generating structured data to drive UI dynamic rendering based on natural language processing

Abstract

The invention discloses a system for generating structured data based on natural language processing to drive UI dynamic rendering. The system comprises a voice acquisition and recognition module, a natural language processing module, a UI dynamic rendering driving module, a control logic automatic binding module and a rendering execution module. User voice or text instructions are automatically analyzed into structured JSON data through natural language processing, Jetpack Compose is driven to dynamically generate UI components, service logic is bound, zero-code interface construction is achieved, in addition, the system is combined with a local AI model, a factory mode and a reflection mechanism, multi-end adaptation, bidirectional data binding and event automatic processing are supported, and the system is suitable for large-scale popularization and application. According to the method, the development threshold is remarkably reduced, the iteration efficiency is improved, the method is particularly suitable for scenes such as smart homes and vehicle-mounted central control which need to quickly customize UIs, and meanwhile offline availability and cross-device consistency are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of UI dynamic rendering technology, and in particular to a system for driving UI dynamic rendering based on structured data generated by natural language processing. Background Technology

[0002] With the development of the Internet of Things and smart terminal devices, more and more smart central control screens (such as home central control panels, smart office touch screens, and in-vehicle large screens) are being deployed in various scenarios, which puts forward higher requirements for the flexibility and adaptability of their interactive interfaces. Jetpack Compose, as a new generation of declarative UI framework on the Android platform, has become a mainstream interface building tool. It has advantages such as high composability and responsive updates, which greatly improves UI development efficiency. However, in practical applications, the use of Jetpack Compose still mainly relies on developers manually coding UI components and binding business logic. This not only results in high development costs and slow response to requirements, but also limits the dynamic generation and personalized adaptation capabilities of the UI. Especially in smart central control screen projects, development teams often need to quickly build interactive interfaces according to the customized requirements of the client. Traditional development methods require frequent code modification and repackaging, resulting in high iteration costs and difficulty in supporting rapid verification of requirements or multi-screen adaptation on the device. At the same time, non-technical personnel such as product managers or operations personnel cannot directly participate in the UI building process, resulting in communication costs and reduced iteration efficiency.

[0003] Furthermore, with the continuous advancement of speech recognition and natural language processing technologies, users can now efficiently express their operational intentions through voice or text. Semantic understanding technology based on NPL can extract structured instructions and interface information from natural language, providing a data foundation for automatic UI generation. However, there is currently a lack of a complete system that can automatically convert structured data into Jetpack Compose UI and simultaneously complete logical binding. Existing solutions either only support UI rendering (such as low-code platforms) or are limited to static configuration, lacking a unified abstraction for dynamic events, state management, and input / output binding. How to adapt to different devices through a unified JSON configuration file and achieve "same description, multi-device adaptation" is also a major pain point in the current development of smart device UIs.

[0004] In summary, this application proposes a system for dynamic UI rendering based on structured data generated by natural language processing. Summary of the Invention

[0005] Based on the technical problems existing in the background technology, this invention proposes a system for dynamic UI rendering driven by structured data generated by natural language processing.

[0006] The system for dynamic UI rendering based on structured data generated by natural language processing proposed in this invention includes a voice acquisition and recognition module, a natural language processing module, a UI dynamic rendering driving module, a control logic automatic binding module, and a rendering execution module.

[0007] The voice acquisition and recognition module is used to acquire natural language voice data input by the user through the microphone, and to use a local voice recognition engine to transcribe the voice into text data in real time.

[0008] The natural language processing module is used to integrate a lightweight intent recognition and entity extraction model, extract key control intent, type, event handling logic, and control structure information, and convert them into JSON format structured data.

[0009] The UI dynamic rendering driver module parses JSON fields and automatically creates UI components through the internal UI factory pattern;

[0010] The automatic binding module for control logic is used to automatically bind user controls to business logic.

[0011] The rendering execution module is used to render the automatically constructed UI tree onto the smart terminal, adapting to different resolutions and device types.

[0012] Preferably, the language acquisition and recognition module understands user intent by deploying a local NLP model. It has the functions of basic instruction recognition, complex intent parsing, and multi-turn dialogue management. Its basic instruction recognition supports direct operations such as opening, closing, and locking the door; complex intent parsing is used to process multi-parameter instructions and extract key information such as the position of the control generation, control logic, and control style; multi-turn dialogue management realizes context association through dialogue state tracking (DST) technology.

[0013] The steps an NLP model takes to understand user intent are as follows:

[0014] S101: Intent Classification: The FastText model is used to quickly determine the domain to which the user's speech belongs;

[0015] S102: Domain Optimization: Load customized dictionaries based on specific scenarios to improve the accuracy of professional instruction recognition;

[0016] S103: Entity Extraction: Using the BERT Tiny+CRF joint model, key parameters of control type, color, position, and logical action are extracted from statements;

[0017] S104: Semantic Normalization: Convert the extracted results into standard fields and standardize their format;

[0018] S105: Structured Generation: Encapsulates the analysis results into a standard JSON configuration format for use by subsequent UI rendering and logic binding modules;

[0019] S106: Instruction parsing and execution: Drives the creation and behavior binding of UI controls based on JSON data, completing the transformation from semantics to a visual interface.

[0020] Preferably, the natural language processing module automatically generates JSON configuration data with a unified structure based on the parsed user intent information. The specific logical steps are as follows:

[0021] S201: Field Mapping: Convert control types, properties, and events into JSON fields;

[0022] S202: Structure Generation: Organize the nested layout of JSON according to the control hierarchy;

[0023] S203: Logic Encapsulation: Encapsulate the event handling logic into JSON using function identifiers or paths;

[0024] S204: Semantic Merging: Merging multiple semantic fragments into a complete component configuration structure;

[0025] S205: Version compatibility: Adapts JSON template structure for different devices.

[0026] Preferably, the UI dynamic rendering driver module is used to receive structured JSON data and drive the dynamic construction of the JetpackCompose component tree, and its specific logical steps are as follows:

[0027] S301: JSON parsing: Parsing component type, style, and layout parameter fields;

[0028] S302: Component construction: Call the internal UI control factory and instantiate the corresponding Compose control through reflection;

[0029] S303: Layout Assembly: Generate Column, Row, and Box layout structures based on the JSON hierarchy, and assemble the nested controls into a complete UI tree;

[0030] S304: Style Adaptation: Set color, font size, and margin UI properties according to the style field;

[0031] S305: Layout Adaptation: Automatically adjusts the size and layout of controls according to the screen size to achieve responsive design.

[0032] Preferably, the specific logical steps of the automatic control logic binding module are as follows:

[0033] S401: Annotation Matching: Match methods annotated with @UIAction based on the event field in the JSON;

[0034] S402: Interface Mapping: Uses an internal interface mapping table to bind control events to a specified ViewModel or controller function;

[0035] S403: DSL Compilation: The event handling logic is described using a DSL, compiled into a Lambda function, and bound to the control.

[0036] S404: Two-way binding: State controls support two-way binding with ViewModel fields to achieve real-time synchronization between UI and data state;

[0037] S405: Event Response: After a click, swipe, or input event is triggered, the bound backend function is automatically called to achieve a closed-loop interaction between the control and the logic.

[0038] Preferably, the specific logical steps of the rendering execution module are as follows:

[0039] S501: Rendering Execution: Execute the generated Compose code on the UI thread and display it in the current screen container;

[0040] S502: State Update: Use Compose's State, LiveData, or Flow objects to manage UI state and enable data changes to drive UI updates;

[0041] S503: Device Adaptation: Dynamically adjusts UI size and layout structure based on device type, resolution, and orientation information to ensure cross-platform consistency;

[0042] S504: Real-time response: When the user interacts with the control, the changes are automatically reflected in the data model, realizing synchronization between the UI and the data;

[0043] S505: Error rollback mechanism: If there is an error while parsing JSON or an event function is not bound, the system will display an error message and load the default UI to ensure that the interface does not crash.

[0044] Compared with existing technologies, the beneficial effects of this invention are:

[0045] 1. By extracting keywords through speech recognition and AI big data models to generate configuration JSON, users can directly perform UI generation and configuration operations through natural language, avoiding traditional manual coding or graphical drag-and-drop methods, improving the intuitiveness of operation, significantly reducing the threshold for use, and making it particularly suitable for non-technical personnel or special groups. Moreover, key instructions can be parsed and executed without relying on cloud servers, making it particularly suitable for running in local device scenarios with weak network, no network, or high privacy requirements, improving system reliability and response speed, and having strong scalability, which can be used in more semantic-driven control scenarios.

[0046] 2. It uses the factory pattern and reflection mechanism to dynamically generate the Compose component tree, eliminating the need for tedious manual UI layout and code debugging, which can significantly shorten the development cycle from requirement change to interface launch, making it suitable for rapid iteration scenarios.

[0047] 3. By combining the UI dynamic rendering driver module and the rendering execution module, it can support unified abstract modeling of component attributes, events, styles, etc., adapt to multiple screen layouts, and has the ability of "same description, multiple terminals reuse", saving the cost of repeated development on multiple terminals; by combining the data binding framework with the event annotation mechanism, it can realize the automatic connection between UI and control logic, and combine with the central control screen embedded system to realize the rapid construction of data-driven multi-terminal UI.

[0048] This invention automatically parses user voice or text commands into structured JSON data through natural language processing, driving Jetpack Compose to dynamically generate UI components and bind business logic, enabling zero-code interface construction. In addition, the system combines local AI models, factory patterns, and reflection mechanisms to support multi-platform adaptation, two-way data binding, and automated event processing, significantly reducing the development threshold and improving iteration efficiency. It is especially suitable for scenarios that require rapid UI customization, such as smart homes and in-vehicle central control systems, while ensuring offline availability and cross-device consistency. Attached Figure Description

[0049] Figure 1 This is a system block diagram of UI dynamic rendering driven by structured data generated through natural language processing, as proposed in this invention.

[0050] Figure 2 This is a flowchart of the system for dynamic UI rendering based on structured data generated by natural language processing, as proposed in this invention. Detailed Implementation

[0051] The present invention will be further explained below with reference to specific embodiments.

[0052] Example

[0053] Reference Figure 1-2This embodiment proposes a system for dynamic UI rendering based on structured data generated by natural language processing, including a voice acquisition and recognition module, a natural language processing module, a UI dynamic rendering driving module, a control logic automatic binding module, and a rendering execution module.

[0054] The voice acquisition and recognition module is used to acquire natural language voice data input by the user through the microphone, and uses a local voice recognition engine to transcribe the voice into text data in real time;

[0055] The language acquisition and recognition module understands user intent by deploying a local NLP model, such as "generate a button." It has the functions of basic instruction recognition, complex intent parsing, and multi-turn dialogue management. Its basic instruction recognition supports direct operations such as opening, closing, and locking doors. Complex intent parsing is used to process multi-parameter instructions, such as "generate a blue button in the center of the screen to control the opening and closing of the curtains," extracting key information such as the position of the control, control logic, and control style. Multi-turn dialogue management achieves contextual association through dialogue state tracking (DST) technology. For example, when the user says "generate the corresponding text prompt below the blue button," the system retrieves the history to fulfill the user's request.

[0056] The steps an NLP model takes to understand user intent are as follows:

[0057] S101: Intent Classification: The FastText model is used to quickly determine the domain to which the user's speech belongs;

[0058] S102: Domain Optimization: Load customized dictionaries based on specific scenarios to improve the accuracy of professional instruction recognition;

[0059] S103: Entity Extraction: Using the BERT Tiny+CRF joint model, key parameters of control type, color, position, and logical action are extracted from statements;

[0060] S104: Semantic Normalization: Convert the extracted results into standard fields and standardize their format;

[0061] S105: Structured Generation: Encapsulates the analysis results into a standard JSON configuration format for use by subsequent UI rendering and logic binding modules;

[0062] S106: Instruction parsing and execution: Drives the creation and behavior binding of UI controls based on JSON data, completing the transformation from semantics to a visual interface;

[0063] The natural language processing module is used to integrate a lightweight intent recognition and entity extraction model, extract key control intent, type, event handling logic, and control structure information, and convert them into JSON format structured data.

[0064] The specific logical steps of the natural language processing module are as follows:

[0065] S201: Field Mapping: Convert control types, properties, and events into JSON fields;

[0066] S202: Structure Generation: Organize the nested layout of JSON according to the control hierarchy;

[0067] S203: Logic Encapsulation: Encapsulate the event handling logic into JSON using function identifiers or paths;

[0068] S204: Semantic merging: Merging multiple semantic fragments into a complete component configuration structure, such as when a user says "a button", "blue", or "place in the middle" respectively;

[0069] S205: Version compatibility: Adapts JSON template structure for different devices, such as central control screens / tablets with different resolutions;

[0070] The UI dynamic rendering driver module parses JSON fields and automatically creates UI components through the internal UI factory pattern;

[0071] The specific logical steps of the UI dynamic rendering driver module are as follows:

[0072] S301: JSON parsing: Parsing component type, style, and layout parameter fields;

[0073] S302: Component Construction: Call the internal UI control factory and instantiate the corresponding Compose control through reflection. The Compose control includes Text, Button, and Slider.

[0074] S303: Layout Assembly: Generate Column, Row, and Box layout structures based on the JSON hierarchy, and assemble the nested controls into a complete UI tree;

[0075] S304: Style Adaptation: Set color, font size, and margin UI properties according to the style field;

[0076] S305: Layout Adaptation: Automatically adjusts the size and layout of controls according to the screen size to achieve responsive design;

[0077] The automatic binding module for control logic is used to automatically bind user controls to business logic.

[0078] The specific logic steps for the automatic binding module to control logic are as follows:

[0079] S401: Annotation Matching: Match methods annotated with @UIAction based on the event field in the JSON;

[0080] S402: Interface Mapping: Uses an internal interface mapping table to bind control events to a specified ViewModel or controller function;

[0081] S403: DSL Compilation: The event handling logic is described using a DSL, compiled into a Lambda function, and bound to the control.

[0082] S404: Two-way binding: State controls support two-way binding with ViewModel fields to achieve real-time synchronization between UI and data state;

[0083] S405: Event Response: After a click, swipe, or input event is triggered, the bound backend function is automatically called to achieve a closed-loop interaction between the control and the logic;

[0084] The rendering execution module is used to render the automatically constructed UI tree onto smart terminals, adapting to different resolutions and device types; the specific logical steps of the rendering execution module are as follows:

[0085] S501: Rendering Execution: Execute the generated Compose code on the UI thread and display it in the current screen container;

[0086] S502: State Update: Use Compose's State, LiveData, or Flow objects to manage UI state and enable data changes to drive UI updates;

[0087] S503: Device Adaptation: Dynamically adjusts UI size and layout structure based on device type, resolution, and orientation information to ensure cross-platform consistency;

[0088] S504: Real-time response: When the user interacts with the control, the changes are automatically reflected in the data model, realizing synchronization between the UI and the data;

[0089] S505: Error rollback mechanism: If JSON parsing fails or an event function is not bound, the system will display an error message and load the default UI to ensure that the interface does not crash;

[0090] This embodiment uses natural language processing to automatically parse user voice or text commands into structured JSON data, driving Jetpack Compose to dynamically generate UI components and bind business logic, achieving zero-code interface construction. In addition, the system combines local AI models, factory patterns, and reflection mechanisms to support multi-terminal adaptation, two-way data binding, and automated event processing, significantly reducing the development threshold and improving iteration efficiency. It is especially suitable for scenarios that require rapid UI customization, such as smart homes and in-vehicle central control systems, while ensuring offline availability and cross-device consistency.

[0091] In this embodiment, the user inputs voice through a microphone (e.g., "Generate a blue, centered button to control the curtains"). The pre-collection and recognition module uses a local speech recognition engine to convert the voice into text, which is then handed over to the natural language processing module for intent classification and entity extraction based on a lightweight intent recognition and entity extraction model. This extracts key information such as control intent, type, event handling logic, and control structure, and converts it into JSON-formatted structured data, which is then output to the UI dynamic rendering driver module. The UI dynamic rendering driver module parses the component type, style, and layout parameters in the JSON using the internal UI factory pattern, dynamically instantiates the corresponding Jetpack Compose component using the UI factory pattern, recursively constructs the layout tree using a reflection mechanism, and automatically adapts the responsive UI according to the device resolution.

[0092] Next, the automatic binding module for control logic scans event fields in the JSON, matches methods annotated with @UIAction or ViewModel functions in the backend, generates Lambda expressions using the DSL compiler to enable hot-plugging of logic, and establishes two-way binding between state controls (Switch / Slider) and the data model. The rendering execution module injects the generated Compose code into the Android view container, listens for user interaction events in real time, triggers the binding logic, and updates the UI state. In case of abnormal situations (such as JSON parsing failure), the system automatically reverts to the default interface to ensure stability.

[0093] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A system for dynamic rendering of structured data driven UI based on natural language processing, characterized in that, The voice collection and recognition module, the natural language processing module, the UI dynamic rendering driving module, the control logic automatic binding module, and the rendering execution module are included. The voice collection and recognition module is used for collecting natural language voice data input by a user through a microphone, and converting the voice into text data in real time by using a local voice recognition engine. The natural language processing module is used for integrating a lightweight intent recognition and entity extraction model, extracting key control intent, type, event processing logic, and control structure information, and converting the information into JSON format structured data. The UI dynamic rendering driving module analyzes JSON fields and automatically creates UI components through an internal UI factory mode. The control logic automatic binding module is used for automatically binding and connecting user controls and business logic. The rendering execution module is used for rendering the automatically constructed UI tree on a smart terminal, and adapting different resolutions and device types. 2.The system for generating dynamic rendering of structured data-driven UI based on natural language processing of claim 1, wherein, The language collection and recognition module understands user intent by deploying a local NLP model, which has the functions of basic instruction recognition, complex intent analysis, and multi-round dialogue management. The basic instruction recognition supports direct operations such as opening and closing doors and locking. The complex intent analysis is used for processing multi-parameter instructions and extracting key information such as the position, control logic, and control style generated by the control. The multi-round dialogue management realizes context association through dialogue state tracking technology. The execution steps of the NLP model understanding user intent are as follows: S101: Intent classification: the FastText model is used to quickly determine the field to which the user voice belongs; S102: Field optimization: a customized dictionary is loaded in combination with a specific scene to improve the accuracy of professional instruction recognition; S103: Entity extraction: the BERT Tiny+CRF joint model is used to extract key parameters such as control type, color, position, and logical action from the sentence; S104: Semantic normalization: the extraction results are converted into standard fields and unified formats; S105: Structured generation: the analysis results are packaged into a standard JSON configuration format for use by subsequent UI rendering and logic binding modules; S106: Instruction analysis and execution: UI control creation and behavior binding are driven according to JSON data to complete the conversion from semantics to visual interface. 3.The system for generating dynamic rendering of structured data-driven UI based on natural language processing of claim 1, wherein, The natural language processing module automatically generates JSON configuration data with a unified structure based on the analyzed user intent information, and the specific logic steps are as follows: S201: Field mapping: control type, attribute, and event are converted into JSON fields; S202: Structure generation: the nested layout of JSON is organized according to the control hierarchical structure; S203: Logic encapsulation: event processing logic is encapsulated into JSON in the form of function identification or path; S204: Semantic merging: multiple semantic segments are merged into a complete component configuration structure; S205: Version compatibility: JSON template structures are adapted for different devices. 4.The system for generating dynamic rendering of structured data-driven UI based on natural language processing of claim 1, wherein, The UI dynamic rendering driving module is used for receiving structured JSON data and driving the dynamic construction of a Jetpack Compose component tree, and the specific logic steps are as follows: S301: JSON parsing: parse component type, style, layout parameter fields; S302: Component construction: call internal UI control factory, instantiate corresponding Compose controls through reflection mechanism; S303: Layout assembly: generate Column, Row, Box layout structure according to JSON hierarchy, and nest controls to assemble complete UI tree; S304: Style adaptation: set color, font size, margin UI properties according to style field; S305: Layout adaptation: automatically adjust control size and layout according to screen size, realize responsive design. 5.The system for generating dynamic rendering of structured data-driven UI based on natural language processing of claim 1, wherein, The specific logic steps of the control logic automatic binding module are as follows: S401: Annotation matching: match methods with @UIAction annotations according to event fields in JSON; S402: Interface mapping: use internal interface mapping table to bind control events to specified ViewModel or control layer functions; S403: DSL compilation: compile event processing logic described by DSL into Lambda function and bind to control; S404: Two-way binding: stateful controls support two-way binding with ViewModel fields, realize real-time synchronization of UI and data state; S405: Event response: after click, slide, input event triggering, automatically call the bound backend function, realize the interaction closed loop of control to logic. 6.The system for generating dynamic rendering of structured data-driven UI based on natural language processing of claim 1, wherein, The specific logic steps of the rendering execution module are as follows: S501: Rendering execution: execute the generated Compose code in UI thread and display it in the current screen container; S502: State update: use Compose's State, LiveData or Flow objects to manage UI state, realize data change driven UI update; S503: Device adaptation: dynamically adjust UI size and layout structure according to device type, resolution, direction information, ensure cross-end consistency; S504: Real-time response: when user interacts with controls, automatically reflect changes to data model, realize UI to data synchronization; S505: Error fallback mechanism: parse JSON exception, event function not bound, system prompts error and loads default UI, ensures interface does not crash.

Images