Staged adaptive voice interaction form mapping generation method and system, and medium

Abstract

The invention provides a mapping generation method and system for a staged adaptive voice interaction form and a medium, and relates to the technical field of man-machine interaction. According to the method, in the development stage, a form UI structure and control semanteme are obtained in a non-intrusive mode through a front-end SDK, a semantic mapping template is automatically generated by utilizing a large language model, and the semantic mapping template is automatically generated; establishing accurate association between the form items and the voice keywords, and embedding the form items and the voice keywords into a host application; and switching to a lightweight mode at the online stage, loading a template, starting a local lightweight text analysis and speech recognition model, efficiently outputting structured field value pairs in combination with user speech input, and driving automatic backfilling, multi-mode prompting or process guidance. Development and online stage decoupling and mapping logic curing are carried out in the development period, a large model or source code modification is not needed after online, the semantic understanding accuracy is guaranteed, meanwhile, resource consumption and the system coupling degree are remarkably reduced, and universality, deployment efficiency and user experience are improved.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a method, system, and medium for mapping and generating voice interaction forms with phased adaptation. Background Technology

[0002] In digital business processes, forms are a fundamental step in information collection. However, traditional manual filling methods, such as keyboard input or screen selection, are inefficient and cumbersome when handling multi-field, long forms, especially for mobile device users or special groups such as the elderly and drivers. To improve this, the industry has tried three types of auxiliary technologies: template filling, OCR recognition, and general voice assistants. However, these solutions all have significant drawbacks: template filling relies on user history data and requires manual pre-setting of field mapping rules, necessitating reconfiguration once the form is modified, resulting in high maintenance costs; OCR technology can only process image information and cannot parse unstructured semantic content spoken, nor can it adapt to complex controls such as single-selection and multiple-selection; general voice assistants have limited functionality, supporting only simple commands and unable to automatically fill in complex forms. Furthermore, existing technologies generally do not differentiate between the different needs of the development and deployment phases. They lack the ability to automatically generate accurate mappings using strong semantic understanding during development and fail to achieve lightweight, low-latency, and efficient operation during deployment, making it difficult to balance universality, accuracy, integration costs, and operational performance in the overall solution. Summary of the Invention

[0003] This application provides a phased adaptation method, system, and medium for generating mappings of voice-interactive forms to solve one or more technical problems existing in the prior art. It provides at least one beneficial option or creates conditions that enable the automatic generation of mapping relationships between form fields and voice commands using a high-precision semantic understanding model during the development phase, and switches to a lightweight model during the deployment phase to achieve low-latency, high-efficiency voice-driven form filling.

[0004] On the one hand, this application provides a method for generating a phased adaptation mapping of voice interaction forms, including: During the development phase, the UI structure and control semantic information of forms in the target application are obtained non-intrusively through the front-end integration SDK; The large language model is invoked to automatically generate a semantic mapping template based on the UI structure and control semantic information, which is then exported and embedded into the host application. The semantic mapping template is used to establish the association between form items and speech-recognizable keywords to support speech field recognition. During the deployment phase, in response to the switch of the runtime configuration to lightweight mode, the semantic mapping template is loaded, and the locally deployed lightweight text parsing model and speech recognition model are enabled. Upon receiving user voice input, the lightweight text parsing model, in conjunction with the semantic mapping template, parses the voice content and outputs structured field-value pairs. These structured field-value pairs are used to drive at least one downstream application, such as automatic form filling, multimodal confirmation prompts, or process guidance. The development phase is decoupled from the deployment phase. The semantic mapping logic is generated and solidified during the development phase, and the deployment phase does not need to rely on a large language model. Furthermore, the entire process does not require modification of the host application's source code.

[0005] Furthermore, the front-end integration SDK is compatible with various front-end technology stacks, including but not limited to Web, H5, mini-programs, and native mobile applications; the non-intrusive acquisition of the UI structure and control semantic information of forms in the target application through the front-end integration SDK includes the following steps: If the target application is a Web, H5, or mini-program application, traverse its Document Object Model (DOM) and extract the form element nodes and their corresponding node attributes. If the target application is a native mobile application, call the accessibility service provided by the operating system to obtain the UI control nodes and their corresponding node attributes in the accessibility tree; Based on the extracted node attributes, the UI structure of the form is constructed, including the parent-child nesting relationship between controls and the spatial layout topology; At the same time, the original semantic clues are obtained from the node attributes, including field label text, placeholder hints, control types, associated description text, and display or selection dependencies between controls; Based on the original semantic clues, the semantic information of the controls corresponding to each form item is determined. The semantic information of the controls includes field labels, control types, and display or selection dependencies between controls.

[0006] Furthermore, the step of invoking the large language model to automatically generate a semantic mapping template based on the UI structure and control semantic information includes the following steps: The UI structure and control semantic information are input into the large language model as context. The pre-defined structured prompts guide the large language model to infer the business semantic role of each form item based on the semantic information of the control, and generate a set of voice-recognizable keywords that match each form item; the set of keywords covers multiple common expressions used by users for the same field in natural language. Based on the control type and interaction constraints of each form item, corresponding field parsing rules are generated; these field parsing rules are used to guide the lightweight text parsing model to perform structured parsing of speech recognition results during the online phase. The keyword set and field parsing rules are integrated to form a standardized semantic mapping template, which is then exported in a structured data format.

[0007] Furthermore, the response to switching the runtime configuration to lightweight mode, loading the semantic mapping template, and enabling the locally deployed lightweight text parsing model and speech recognition model includes the following steps: Read the runtime configuration parameters and determine whether the current stage is the deployment stage; During the deployment phase, the semantic mapping template is loaded from the host application's local storage, and its integrity and version validity are verified. Initialize a professional speech recognition model, which is adapted to the form filling scenario and enables endpoint detection and hot word enhancement functions; A quantized and compressed lightweight text parsing model is loaded, which is deployed on a terminal device or edge server for operation in an environment without network connection or where large language models are disabled; wherein, the model compression process includes at least one of quantization, pruning, or knowledge distillation. The speech recognition model and the lightweight text parsing model are bound to the runtime engine of the front-end integration SDK to respond to user voice input and perform field parsing.

[0008] Furthermore, the semantic mapping template supports versioned hot updates after digital signature verification via a secure channel during the deployment phase, specifically including: The update packet of the semantic mapping template is received through a secure channel; Verify the digital signature and version compatibility of the update package; Replace the currently loaded semantic mapping template at runtime without restarting the host application or redeploying the installation package.

[0009] Furthermore, when parsing speech content, the lightweight text parsing model performs context-aware semantic disambiguation based on the currently identified field values ​​and the form business context to distinguish multiple form items with similar semantics.

[0010] Furthermore, when voice parsing fails or no field can be matched, a fallback strategy is triggered. The fallback strategy includes prompting the user to repeat the statement, switching to manual input mode, or logging a miss log with the user's authorization, which can be used to optimize the semantic mapping template in subsequent development stages.

[0011] Furthermore, while generating semantic mapping templates, the large language model automatically generates a speech-text simulation sample set that covers the diversity of user expressions. The speech-text simulation sample set is used to verify the mapping coverage or fine-tune the lightweight text parsing model.

[0012] On the other hand, this application provides a phased adaptation system for mapping and generating voice interaction forms, the system comprising: The form parsing module is configured to non-intrusively obtain the UI structure and control semantic information of forms in the target application during the development phase through the front-end integration SDK; The mapping generation module is configured to call a large language model to automatically generate a semantic mapping template based on the UI structure and control semantic information, export it and embed it into the host application; the semantic mapping template is used to establish the association between form items and speech-recognizable keywords to support speech field recognition. The running switching module is configured to, during the online phase, in response to the running configuration switching to lightweight mode, load the semantic mapping template and enable the locally deployed lightweight text parsing model and speech recognition model. The voice parsing module is configured to, upon receiving user voice input, use the lightweight text parsing model in conjunction with the semantic mapping template to parse the voice content and output structured field-value pairs; the structured field-value pairs are used to drive at least one downstream application among automatic form filling, multimodal confirmation prompts, or process guidance; The development phase is decoupled from the deployment phase. The semantic mapping logic is generated and solidified during the development phase, and the deployment phase does not need to rely on a large language model. Furthermore, the entire process does not require modification of the host application's source code.

[0013] On the other hand, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for generating a phased adaptation voice interaction form.

[0014] The beneficial effects of this application are as follows: This application provides a phased adaptation method for generating mappings of voice-interactive forms. During the development phase, it utilizes a front-end integration SDK to non-intrusively obtain the UI structure and control semantic information of the target application's forms. A large language model is then used to automatically generate a semantic mapping template, establishing precise associations between form items and voice-recognizable keywords. This template is then embedded into the host application. During the deployment phase, a lightweight mode is switched to, loading the generated semantic mapping template and enabling a locally deployed lightweight text parsing model and speech recognition model. Upon receiving user voice input, the template is used to efficiently parse structured field value pairs, which are then used to drive downstream applications such as automatic form backfilling, multimodal confirmation prompts, or process guidance. The entire process decouples the development and deployment phases. The semantic mapping logic is solidified during development, and after deployment, it does not rely on a large language model or require modification of the host application's source code. This significantly reduces runtime resource consumption and system coupling while ensuring high-accuracy semantic understanding, improving the versatility, deployment efficiency, and user experience of voice-interactive forms. This application also provides a corresponding system and medium, the beneficial effects of which are similar to the method and will not be elaborated upon here.

[0015] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0016] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0017] Figure 1 This is a flowchart of the method for generating a phased adaptation voice interaction form provided in this application; Figure 2 This is a structural diagram of the phased adaptation voice interaction form mapping generation system provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] In the context of rapid digital and intelligent development, forms, as the most basic and widespread information input medium in human-computer interaction, are widely used in various business scenarios such as government services, financial account opening, medical consultations, e-commerce ordering, and enterprise office work. Users typically need to manually fill in large amounts of structured or semi-structured data on web pages or mobile applications, including text input, single-choice and multiple-choice, date selection, drop-down list controls, and other operations. However, this traditional interaction mode has significant pain points. For ordinary users, especially in the small-screen environment of mobile devices, the process of frequently switching input methods, clicking on each option, and repeatedly proofreading content is cumbersome and time-consuming. For special groups such as the elderly, visually impaired people, or users who are driving, manual filling is almost impractical, severely restricting the accessibility and inclusiveness of digital services.

[0023] To improve form completion efficiency and user experience, the industry has tried various auxiliary technologies. The first type is autofill technology based on historical data or rule templates. This method records the user's past entries and performs field matching and backfilling when similar forms appear. However, this type of solution is highly dependent on existing user data and is ineffective for new users or first-time form completion scenarios. More importantly, its field mapping relationship needs to be manually preset or inferred through simple rules. Once the form structure changes (such as adjusting the field order, fine-tuning the label text, or changing the control type), the original mapping becomes invalid, resulting in extremely high maintenance costs and difficulty in generalizing to forms in different business domains.

[0024] The second category is based on Optical Character Recognition (OCR) technology. This method involves photographing paper forms or screenshots, using OCR to extract the text content from the image, and attempting to reconstruct it into a structured form. However, OCR can only process static visual information, cannot understand semantic context, and is even less capable of handling unstructured speech content uttered by the user. Furthermore, OCR is sensitive to image quality, with accuracy dropping significantly under complex layouts, handwritten fonts, or low-light conditions. In addition, OCR outputs a raw text stream, lacking the ability to perceive form control types (such as radio buttons, checkboxes, and dates), making it difficult to directly drive the automatic filling of interactive forms.

[0025] The third category is general-purpose voice assistant integration solutions, such as calling a smart speaker or the built-in voice assistant on a mobile phone to execute commands like "enter Zhang San in the name field." However, these general-purpose assistants lack an understanding of the specific application interface structure and cannot accurately locate the target form item; their voice commands must strictly follow preset syntax, resulting in poor flexibility; and they usually only support simple operations and cannot handle complex logic (such as dynamically jumping to branches based on voice content or linking multiple fields); more importantly, general-purpose voice assistants mostly rely on large cloud models, resulting in high response latency, strong network dependence, and significant privacy risks, making it difficult to meet the performance, security, and stability requirements of enterprise-level applications.

[0026] A deeper problem lies in the fact that existing technologies generally fail to differentiate between the different needs of the application development phase and the product deployment phase. During development, developers urgently need a capability to deeply understand UI semantics and automatically generate high-precision field mappings to reduce configuration costs. However, during deployment, the system prioritizes low latency, low resource consumption, and high reliability, making it unsuitable for continuously calling computationally intensive large language models. Current solutions either rely entirely on heavy models, leading to deployment difficulties, or use static rules that sacrifice accuracy, failing to achieve a seamless integration of intelligent generation during development and efficient execution during runtime. Furthermore, most solutions require modifying the host application's source code to inject recognition logic, which not only increases integration complexity but may also compromise the stability of existing systems, hindering the rapid adoption of the technology in existing applications.

[0027] In summary, existing voice-interactive form technologies have significant shortcomings in terms of universality, accuracy, deployment flexibility, operational efficiency, and intrusiveness to the host system. There is an urgent need for a new solution that can automatically generate accurate mappings during the development phase using advanced semantic understanding capabilities, and can also run in a lightweight and efficient manner after deployment without modifying the original application code, and support cross-platform deployment.

[0028] To address the aforementioned issues, this application proposes a phased adaptation method for generating voice-interactive form mappings. During the development phase, a front-end integrated SDK is used to automatically capture the UI structure and semantic information of the target application's forms in a non-intrusive manner. A large language model is then invoked to automatically generate a precise semantic mapping template based on this information, establishing a structured association between form items and voice-recognizable keywords. This template is subsequently embedded into the host application. During the deployment phase, the system automatically switches to a lightweight operating mode, loading a locally deployed lightweight text parsing model and speech recognition model. Combined with the pre-defined semantic mapping template, the system parses the user's voice input in real time and outputs structured field value pairs to drive downstream functions such as automatic form backfilling, multimodal confirmation, or process guidance. The entire solution decouples high-precision semantic modeling during development from low-resource, high-efficiency execution during deployment. It does not rely on external large models or require modification of the host application's source code, significantly improving the universality, deployment convenience, operational performance, and user adaptability of voice-interactive forms.

[0029] The front-end integration SDK is a lightweight software development kit embedded in the host application's front-end code. It is defined as a collection of modular components that can run without modifying the host application's core logic or source code. Its main purpose is to non-intrusively capture the UI structure and control semantic information of form interfaces during the development phase, and to serve as the runtime engine for voice interaction functions during the deployment phase. Its significance lies in achieving unified perception and intelligent interaction support for forms across multiple platforms, including Web, H5, mini-programs, and native mobile applications. It assists in generating accurate semantic mapping templates during development and efficiently schedules local lightweight models to complete voice parsing and field backfilling during runtime. This ensures the stability of the host application while seamlessly endowing it with voice-driven form filling capabilities, significantly reducing integration costs and improving cross-platform compatibility and user experience.

[0030] This application employs an ASR (Automatic Speech Recognition) model as its automatic speech recognition model, an intelligent algorithm system that converts user-input speech signals into corresponding text content in real time. In this application, the model undergoes domain adaptation for form filling scenarios and possesses capabilities such as endpoint detection, noise reduction, and hot word enhancement, enabling it to accurately recognize the natural language commands issued by the user during form filling. Its significance lies in serving as the front-end perception entry point for voice interaction, efficiently converting the user's colloquial expressions into structured text, providing reliable input for subsequent lightweight text parsing models, thereby supporting the entire voice-driven automatic form filling process. Simultaneously, the model supports local deployment and offline operation, ensuring low-latency, highly available speech recognition services even in environments without network access or with privacy concerns.

[0031] First, the method for generating voice interaction form mapping in stages according to the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0032] Reference Figure 1 The implementation process of the phased adaptation voice interaction form mapping generation method provided in this application embodiment includes, but is not limited to, the following steps.

[0033] Step S110: During the development phase, the UI structure and control semantic information of the forms in the target application are obtained non-intrusively through the front-end integration SDK.

[0034] In step S110, during the development phase, the UI structure and control semantic information of the forms in the target application are obtained non-intrusively through the SDK integrated into the front end. This step does not modify the source code of the host application, but rather utilizes the SDK to dynamically scan and extract semantic features such as the hierarchical structure, control types, label text, placeholders, and relationships of the form interface at runtime, thereby constructing a structured interface description that can be used for subsequent processing. This process provides basic data support for the voice interaction system to understand the form content, ensuring that subsequent mapping generation can accurately align user voice commands with specific form items.

[0035] Step S120: Call the large language model to automatically generate a semantic mapping template based on the UI structure and control semantic information, export it and embed it into the host application.

[0036] The semantic mapping template is used to establish the association between form items and voice-recognizable keywords to support voice field recognition.

[0037] In step S120, the large language model is invoked to automatically generate a semantic mapping template based on the UI structure and control semantic information obtained in step S110. This template is then exported and embedded into the host application. Leveraging its powerful contextual understanding and natural language generalization capabilities, the large language model can infer diverse expressions that users might use from control semantics and establish many-to-many associations between form items and speech-recognizable keywords. The generated semantic mapping template solidifies these mapping rules in a structured form, serving as a bridge between speech recognition and form filling, laying a semantic foundation for efficient parsing during the deployment phase.

[0038] In step S130, during the deployment phase, in response to the switch of the running configuration to lightweight mode, the semantic mapping template is loaded, and the locally deployed lightweight text parsing model and speech recognition model are enabled.

[0039] In step S130, during the deployment phase, the system automatically switches to lightweight mode based on the runtime configuration, loads the semantic mapping templates generated and embedded during the development phase, and simultaneously enables the locally deployed lightweight text parsing model and speech recognition model. This step signifies the system's transition from a high-precision modeling phase to a low-resource operation phase, no longer relying on large language models or network connections, but instead using pre-built mapping templates and lightweight local models to complete real-time speech processing. This mode switch effectively reduces runtime computational overhead, response latency, and privacy risks, while ensuring availability in environments without or with weak network connectivity.

[0040] Step S140: Upon receiving user voice input, the lightweight text parsing model, combined with a semantic mapping template, parses the voice content and outputs structured field-value pairs.

[0041] The structured field-value pairs are used to drive at least one downstream application, such as form autofill, multimodal confirmation prompts, or process guidance.

[0042] In step S140, upon receiving user voice input, a lightweight text parsing model, combined with a loaded semantic mapping template, performs semantic parsing on the speech recognition result, outputting structured field-value pairs. This step accurately maps the user's natural language speech content to specific form fields and extracts the corresponding fill values, forming structured data that can be directly processed by the machine. The output field-value pairs can be directly used by downstream functional modules to drive automatic form filling, send multimodal confirmation prompts to the user, or trigger subsequent business process guidance, thereby achieving an end-to-end voice-interactive form filling experience.

[0043] In some embodiments of this application, the development phase and the deployment phase are decoupled, separating the generation and operation of semantic mapping logic; the semantic mapping logic is solidified and generated during the development phase, and the deployment phase does not need to rely on a large language model, and the entire process does not require modification of the host application source code.

[0044] Specifically, during the development phase, the system utilizes a large language model to understand the semantics of forms and construct mapping rules, solidifying the results into semantic mapping templates. After product launch, the system no longer needs to call or connect to any large language model; it can complete speech parsing tasks solely based on the embedded templates and local lightweight models. This decoupled design significantly reduces post-launch computational resource consumption, network dependencies, and response latency, while simultaneously improving system stability and privacy security. Furthermore, the entire process achieves non-intrusive integration with the host application through a front-end SDK, requiring no modification to the host application's source code. This greatly simplifies the deployment process, lowers the technical barrier to entry, and enables the solution to quickly adapt to various existing or new applications, demonstrating strong engineering feasibility and promotional value.

[0045] In some embodiments of this application, the front-end integration SDK adapts to various front-end technology stacks, including but not limited to Web, H5, mini-programs, and native mobile applications. In step S110, during the development phase, the front-end integration SDK non-intrusively obtains the UI structure and control semantic information of forms in the target application, including the following steps.

[0046] Step S210: If the target application is a Web, H5, or mini-program application, traverse its Document Object Model (DOM) and extract form element nodes and their corresponding node attributes.

[0047] In step S210, for Web, H5, or mini-program applications, form element nodes and their corresponding node attributes are extracted during the development phase by traversing their Document Object Model (DOM). This step utilizes the DOM access capabilities natively supported by the front-end runtime environment to systematically scan form-related elements on the page, such as input boxes, dropdown menus, and checkboxes, and obtain attribute information such as the tag name, class name, ID, text content, placeholders, and relationships of each node. This process provides the raw data foundation for subsequently building a structured UI description and understanding control semantics, ensuring that the form's interface structure is fully captured without modifying the application code.

[0048] Step S220: If the target application is a native mobile application, call the accessibility service provided by the operating system to obtain the UI control nodes and their corresponding node attributes in the accessibility tree.

[0049] In step S220, when the target application is a native mobile application, the UI control nodes and their attributes in the accessibility tree are obtained by calling the accessibility services provided by the operating system. Since native applications do not expose a DOM-like structure, this step leverages system-level interfaces such as Android's Accessibility API or iOS's UI Automation to traverse the interface controls in an accessible manner, extracting accessibility attributes including control type, text labels, tooltips, and parent-child relationships. This approach achieves non-intrusive awareness of the native application's form structure, enabling this solution to cover mainstream mobile technologies across platforms.

[0050] Step S230: Based on the extracted node attributes, construct the UI structure of the form. The UI structure includes the parent-child nesting relationship between controls and the spatial layout topology.

[0051] In step S230, the UI structure of the form is constructed based on the node attributes extracted from the DOM or accessibility tree. This structure clearly expresses the parent-child nesting relationships and spatial layout topology between controls. By analyzing the hierarchical paths, coordinate positions, and visual groupings of the nodes, the system reconstructs the actual organization of the form on the user interface, such as which input boxes belong to the same block and which options are related to each other. This structured representation not only preserves the logical hierarchy of the interface but also provides contextual information for subsequent semantic parsing, serving as a crucial intermediate product connecting interface presentation and semantic understanding.

[0052] Step S240: Obtain the original semantic clues from the node attributes. The original semantic clues include field label text, placeholder hints, control types, associated description text, and display or selection dependencies between controls.

[0053] In step S240, raw semantic cues are obtained from the extracted node attributes. These cues include field label text, placeholder hints, control types, associated description text, and display or selection dependencies between controls. This step focuses on identifying textual and interactive features directly related to the user's understanding of the form's intent, such as the "Name" label, the "Please enter your phone number" placeholder, dropdown selection controls, the adjacent question mark icon, or conditional logic like "If you select 'Other,' please fill in the reason." These raw semantic cues are the core basis for inferring the correspondence between the user's voice expression and form items, providing semantic input for subsequently generating accurate mappings.

[0054] Step S250: Based on the original semantic clues, determine the control semantic information corresponding to each form item. The control semantic information includes field labels, control types, and display or selection dependencies between controls.

[0055] In step S250, based on the aforementioned original semantic clues, the semantic information of the controls corresponding to each form item is determined. This information specifically includes field labels, control types, and display or selection dependencies between controls. This step summarizes and structures the original clues, integrating scattered attributes into a complete semantic description of each form item. For example, an input box, its left-hand "email" label, the "type" attribute of "email," and the logical rule "required only during registration" are unified into a single semantic unit. This structured control semantic information can be directly used by a large language model to understand the form's intent and is a prerequisite for automatically generating high-quality semantic mapping templates.

[0056] In some embodiments of this application, step S120 involves calling a large language model to automatically generate a semantic mapping template based on the UI structure and control semantic information, including the following steps.

[0057] Step S310: Input the UI structure and control semantic information as context into the large language model.

[0058] Step S310 serves to provide the UI structure and control semantic information acquired during the development phase as contextual input to the large language model. This step aims to build a complete form understanding environment for the large language model, enabling it to fully grasp the position and function of each form item within the business scenario based on multi-dimensional semantic cues such as the actual interface hierarchy, control types, label text, placeholders, and dependencies. By inputting this structured and semi-structured front-end information into the model in natural language or intermediate representation, an accurate and rich contextual foundation is laid for its subsequent inference of field semantics and generation of mapping rules.

[0059] Step S320 involves guiding the large language model to infer the business semantic role of each form item based on the semantic information of the controls, using preset structured prompts. A set of voice-recognizable keywords matching each form item is then generated. This keyword set covers various common expressions used by users for the same field in natural language.

[0060] In step S320, pre-set structured prompts guide the large language model to infer the business semantic role of each form item based on the semantic information of the controls, and generate a set of speech-recognizable keywords matching each form item. This step utilizes prompt engineering to enable the large language model to focus on understanding business meanings such as "Does this input box represent a delivery address or an invoice header?" and "Does this dropdown option involve an identity type?" Based on this, it simulates the diverse expressions that users might use in real voice interactions, such as generating synonyms or near-synonyms for the "phone number" field like "telephone," "contact number," "mobile phone," and "mobile number." The generated keyword set covers multiple common expressions for the same field in natural language, significantly improving the robustness of speech recognition and the inclusiveness of user expression.

[0061] Step S330: Based on the control type and interaction constraints of each form item, generate corresponding field parsing rules. These field parsing rules guide the lightweight text parsing model in performing structured parsing of the speech recognition results during the deployment phase.

[0062] In step S330, corresponding field parsing rules are generated based on the control type and interaction constraints of each form item. This step focuses on how to correctly convert the text results after speech recognition into structured values ​​that meet the form requirements. For example, for date controls, the parsing rules may require standardizing "next Monday" or "March 5th next year" into YYYY-MM-DD format; for radio button groups, the rules need to limit the output value to one of the predefined options; for fields with linkage logic, conditional judgment rules need to be included. These parsing rules provide clear processing guidance for the lightweight text parsing model in the online phase, ensuring that speech content can be accurately and compliantly mapped into data that can be directly used for form filling.

[0063] Step S340: Integrate the keyword set and field parsing rules to form a standardized semantic mapping template, and export it in a structured data format.

[0064] In step S340, the keyword set and field parsing rules generated in the previous steps are integrated to form a standardized semantic mapping template, which is then exported in a structured data format. This step completes the transformation from model output to deployable assets, organizing the scattered semantic mapping elements into a unified, machine-readable template file, such as JSON or Protocol Buffer format. Each form item in the template is associated with its keyword set and parsing rules, resulting in a clear overall structure that is easy to load and parse. The exported semantic mapping template is then embedded into the host application, serving as the core configuration basis for achieving efficient voice-driven form filling during the deployment phase, thus achieving seamless transfer of intelligent generation results during development to execution logic during runtime.

[0065] In some embodiments of this application, in step S130, during the online phase, in response to switching the running configuration to lightweight mode, the semantic mapping template is loaded, and the locally deployed lightweight text parsing model and speech recognition model are enabled, including the following steps.

[0066] Step S410: Read the running configuration parameters and determine whether the current stage is the online stage.

[0067] In step S410, the runtime configuration parameters are read to determine whether the current stage is the deployment stage. This step checks the configuration flags preset or dynamically issued during application deployment to determine that the system should currently enter a lightweight runtime mode, thereby triggering a different execution path than the development stage. This determination is a key control point for decoupling the development and deployment stages, ensuring that the system automatically switches to a low-resource, high-efficiency runtime strategy after the product is officially released, avoiding the misuse of heavyweight models or debugging logic relied upon during the development phase.

[0068] Step S420: During the deployment phase, load the semantic mapping template from the host application's local storage and verify its integrity and version validity.

[0069] In step S420, the semantic mapping template is loaded from the host application's local storage, and its integrity and version validity are verified. This step ensures that the mapping rules generated and embedded during the development phase can be accurately read during the deployment phase. Integrity verification prevents the template from being tampered with or corrupted, while version validity checks prevent parsing errors caused by mismatches between the template and the current form structure. This process ensures the stability and reliability of the voice interaction function across different application version iterations and is a crucial link connecting development results with runtime behavior.

[0070] Step S430: Initialize the professional speech recognition model. The speech recognition model is adapted to the domain for the form filling scenario, and endpoint detection and hot word enhancement functions are enabled.

[0071] In step S430, a professional speech recognition model is initialized. This model is domain-adapted for form filling scenarios and enables endpoint detection and hot word enhancement. By introducing a large amount of form-related speech data during the training phase, the model achieves higher recognition accuracy for words, sentence structures, and pronunciation habits commonly used by users during the filling process. The endpoint detection function can accurately determine the start and end points of the user's speech, improving the interaction response speed. Hot word enhancement dynamically injects keywords from the semantic mapping template into the recognition engine, significantly improving the recognition recall rate of key field words. This step provides foundational support for subsequent high-quality speech-to-text conversion.

[0072] Step S440: Load the quantized and compressed lightweight text parsing model. The lightweight text parsing model is deployed on a terminal device or edge server for operation in environments without network connectivity or where large language models are disabled. The model compression process includes at least one of quantization, pruning, or knowledge distillation.

[0073] In step S440, a quantized and compressed lightweight text parsing model is loaded. This model is deployed on a terminal device or edge server for operation in environments without network connectivity or where large language models are disabled. The model compression process employs at least one technique among quantization, pruning, or knowledge distillation to significantly reduce model size and computational requirements while preserving as much of the understanding and parsing capabilities of semantic mapping templates as possible. This step ensures that the system can independently convert speech content into structured field values ​​even on resource-constrained mobile devices or in privacy-sensitive scenarios, without relying on cloud services, thus improving system availability, response speed, and data security.

[0074] Step S450: Bind the speech recognition model and the lightweight text parsing model to the runtime engine of the front-end integration SDK to respond to user voice input and perform field parsing.

[0075] In step S450, the speech recognition model and the lightweight text parsing model are bound to the runtime engine of the front-end integrated SDK to respond to user voice input and perform field parsing. This step completes the integration of the model and the interaction entry point, enabling the SDK to automatically call the speech recognition model to obtain the text result when the user triggers voice input, and immediately hand it over to the lightweight text parsing model for structured parsing in conjunction with the semantic mapping template. By unifying the two models under the SDK runtime management, an end-to-end pipeline from voice acquisition to field output is realized, providing real-time and reliable structured data input for downstream form filling or multimodal prompts.

[0076] In some embodiments of this application, a form mapping structure generation and configuration scheme based on a large model is provided for efficiently building standardized and highly compatible semantic mapping templates during the development phase. This scheme preferentially uses large language models with strong structured understanding capabilities, such as OpenAI GPT-4, Anthropic Claude 3, or Google Gemini Pro, to ensure accurate parsing and mapping generation of complex form UI structures (including nested controls and dynamic fields).

[0077] In practical implementation, the SDK provides an encrypted configuration panel. Developers input the large model API key through this interface. The key is stored in environment variables and only takes effect in the development environment, not packaged with the SDK into the production version, thus ensuring key security and preventing plaintext leakage. To optimize model output quality, the SDK sets the model parameters by default: the temperature value is set to 0.2-0.3 to reduce the randomness of the generated results and improve the standardization of the mapping structure; the maximum number of tokens is set to 2048 to meet the processing requirements of long forms with more than 30 fields. The system prompt explicitly requires the model to analyze the form UI structure and output JSON format data containing field_label (field name), control_type (control type, such as INPUT_TEXT, RADIO, SELECT, CHECKBOX), and options (optional content). The field names must use natural language that users can understand (such as "phone number" instead of "tel_input_123") to ensure the understandability of subsequent speech recognition.

[0078] During the rule training phase, developers upload 3 to 5 sample forms covering different control types. The SDK calls the large model to learn the correspondence between "UI elements → mapping fields." For example, it identifies controls with the label text "Gender" and the type "radio" as {"field_label":"Gender","control_type":"RADIO"}, achieving automated learning of general rules. Subsequently, the SDK triggers the large model to parse the target form, generating an initial mapping structure and providing a visual preview interface for developers to manually correct field names or control types to ensure mapping accuracy. After successful verification, the SDK exports the final mapping structure as a form_mapping_template.json file and automatically embeds it into the SDK's resource directory during the deployment phase. This allows for direct use in runtime speech parsing and field matching without secondary development, achieving a seamless transition from development to deployment.

[0079] In some embodiments of this application, a form semantic mapping data structure automatically generated by a large language model is used as an example. This mapping structure is organized in JSON format and includes a unique form identifier (form_id), version number, and a list of fields. Each field defines its semantic attributes and voice matching rules. For example, the field "Name" is marked as INPUT_TEXT type, and its match_keywords include common user expressions such as "Name", "My name is", and "My name is", achieving accurate recognition of various forms of address in natural language; the "Gender" field is defined as RADIO type and lists the options "Male" and "Female", while associating the keywords "Gender", "Male", and "Female", supporting selection operations in voice commands; "Insurance Type" is a SELECT control, containing two options: "Employee Social Security" and "Resident Social Security", and its keyword set covers "Insurance Type" and specific option names, improving matching accuracy; "Insurance Additional Items" is a CHECKBOX type, supporting multiple selections, and its options include "Medical Insurance" and "Unemployment Insurance", with "Medical Insurance" automatically added by the large model as a synonym, enhancing the robustness of voice understanding. This structure achieves semantic alignment from UI elements to voice commands by standardizing field labels, control types, optional items, and voice keywords. This provides a reliable basis for efficient and accurate field matching and value filling in the subsequent lightweight text parsing model at runtime, significantly improving the versatility and user experience of voice interaction forms.

[0080] In some embodiments of this application, an automatic model operation mode switching mechanism based on the SDK's built-in "stage switching module" is provided to achieve seamless transition between the development and deployment phases. After the semantic mapping template is generated and validated during the development phase, developers only need to set the `stage` parameter to "online" in the SDK configuration to trigger the SDK's internal operation mode switching logic. The system automatically switches from the development mode, which relies on a large language model, to a lightweight "small model + ASR" operation mode. This mechanism does not require modification of any source code in the host application, ensuring non-intrusive technology integration and convenient deployment. After the switch is complete, the SDK automatically loads the `form_mapping_template.json` file exported during the development phase and uses it as the basis for runtime semantic parsing. Simultaneously, it enables the locally deployed lightweight text parsing model and professional ASR model to achieve real-time processing and field matching of user voice input. Through this automated switching process, the system can smoothly transition from high-precision modeling to efficient execution without interrupting business logic, ensuring the consistency, stability, and maintainability of the form voice interaction function across different lifecycle stages.

[0081] In some embodiments of this application, a closed-loop process for voice-interactive form filling is provided, realizing end-to-end intelligent processing from voice input to automatic backfilling. The process begins with the voice acquisition and ASR conversion stage. The user clicks a floating button provided by the SDK to trigger the microphone to acquire voice. The SDK then calls a locally deployed professional ASR model (such as iFlytek ASR) to convert the voice content into text in real time, for example, "My name is Li Ming, male, applying for employee social security and supplementary medical insurance," and displays this to the user visually for confirmation. Manual correction of typos is supported to ensure input accuracy.

[0082] The next stage involves text parsing and field matching in a small model. The SDK inputs the confirmed text along with the `form_mapping_template.json` mapping template generated during development into the lightweight text parsing model. This model performs semantic parsing on the text based on the `match_keywords` list for each field in the template, extracting structured "field-value" pairs, such as "Name → Li Ming" and "Supplementary Insurance Items → Medical Insurance". For option-type controls (such as CHECKBOX), the model supports fuzzy matching; when the user says "medical insurance," it can accurately match the "medical insurance" option, improving the naturalness of the interaction and its error tolerance.

[0083] Finally, the automated UI backfill and result feedback stage begins. The SDK iterates through all successfully matched "field-value" pairs and performs corresponding operations based on the control_type of each field: directly filling in text for INPUT_TEXT type fields; simulating clicking the corresponding option for RADIO or SELECT type fields; and batch selecting matching items for CHECKBOX type fields. After backfilling is complete, the SDK highlights the filled fields and marks unmatched fields (such as "emergency contact") with a red box, while a pop-up window prompts "Please complete the missing item: emergency contact," guiding the user to complete the subsequent filling. The entire process forms a complete closed loop of voice collection, recognition, parsing, backfilling, and feedback, significantly improving form filling efficiency and user experience.

[0084] In some embodiments of this application, in step S130, the semantic mapping template supports versioned hot updates after digital signature verification through a secure channel during the online phase, specifically including the following steps.

[0085] First, by receiving semantic mapping template update packages through a secure channel, the system can dynamically obtain the latest mapping rules after the product goes live, without relying on app store approval or manual user upgrades. The secure channel ensures data integrity and tamper-proofing during transmission, preventing malicious man-in-the-middle injection of illegal templates, thus enabling flexible content maintenance and business adaptation while ensuring security.

[0086] Secondly, verify the digital signature and version compatibility of the update package. Before loading a new template, confirm its credible origin and compatibility with the current host application and runtime environment. Digital signature verification ensures that the update package is released by an authorized party and has not been tampered with; version compatibility checks prevent parsing errors or functional abnormalities caused by changes in template structure or inconsistent field definitions, thus maintaining system stability and functional correctness during dynamic updates.

[0087] Furthermore, by replacing the currently loaded semantic mapping template at runtime, seamless hot updates of semantic mapping logic can be achieved without restarting the host application or republishing the installation package. Once verified, the new template takes effect immediately and can be used for subsequent speech parsing tasks. Users experience no change, and developers do not need to repackage the application or force users to update it. This mechanism greatly improves operational efficiency and business response speed, and is especially suitable for scenarios where form structures are frequently adjusted or mapping errors need to be quickly fixed.

[0088] In some embodiments of this application, in step S140, when the lightweight text parsing model parses the speech content, it performs context-aware semantic disambiguation based on the currently identified field values ​​and the form business context to distinguish multiple form items with similar semantics, thereby solving the problem of field matching ambiguity caused by word ambiguity or polysemy in the user's speech expression.

[0089] When a user utters a brief expression such as "Fill in Zhang San" or "Select Beijing," the system may encounter situations where multiple form fields (such as name, emergency contact, departure point, and destination) can match the same keyword. In this case, the model dynamically determines the most likely target field by combining the already filled field values ​​in the current form with the overall business scenario (e.g., whether the user is currently on an order form or a user registration page), thus accurately mapping the voice content to the correct form field. This context-aware capability significantly improves the accuracy and robustness of voice parsing, avoiding errors or interaction failures caused by confusion due to semantically similar fields, and making voice interaction closer to the natural expression habits of real users.

[0090] In some embodiments of this application, in step S140, when the speech parsing fails or no field can be matched, a fallback strategy is triggered. The fallback strategy includes: prompting the user to repeat, switching to manual input mode, or recording a miss log with the user's authorization, for use in subsequent development stages to optimize the semantic mapping template.

[0091] Specifically, triggering a fallback strategy when speech parsing fails or cannot match any field ensures the continuity of user experience and system availability. When the lightweight text parsing model cannot determine the target field due to speech recognition errors, user expressions exceeding the preset keyword range, or incomplete semantic mapping coverage, the system will not become unresponsive but will proactively activate preset response mechanisms. Prompting the user to restate provides a second chance to express themselves, improving the success rate of interaction; switching to manual input mode ensures that the user can always complete the form, avoiding functional blockage; and logging the misses with user authorization provides feedback data from real-world scenarios for subsequent development phases, used to analyze mapping blind spots and optimize semantic mapping templates. This fallback strategy balances runtime robustness with long-term iterative evolution, enabling the system to maintain reliable service even when facing complex and ever-changing user language behaviors.

[0092] In some embodiments of this application, the large language model automatically generates a speech-text simulation sample set that covers the diversity of user expressions while generating semantic mapping templates. The speech-text simulation sample set is used to verify the mapping coverage or fine-tune the lightweight text parsing model.

[0093] Specifically, the role of the large language model in automatically generating a speech-text simulation sample set covering the diversity of user expressions while generating semantic mapping templates is to provide high-quality, high-coverage training and testing data for subsequent verification and optimization. Based on its understanding of the semantics of form items, the large language model simulates the diverse natural language expressions that real users may use in different regions, ages, habits, or contexts, generating a large number of speech-to-text samples corresponding to each field. These samples not only cover standard expressions but also complex forms such as colloquialisms, abbreviations, synonym substitutions, and dialect variations, thus comprehensively reflecting the language diversity in actual use. The generated speech-text simulation sample set can be directly used to evaluate the coverage of various expressions by the current semantic mapping template and discover potential mapping blind spots; at the same time, it can also be used as supervisory data to fine-tune the lightweight text parsing model, improving its accuracy in recognizing and parsing non-standard expressions during the deployment phase, thereby enhancing the generalization ability and user experience consistency of the entire voice interaction system.

[0094] Secondly, refer to Figure 2 This application provides a phased adaptation system for mapping and generating voice-interactive forms. The system includes a form parsing module, a mapping generation module, a running switching module, and a voice parsing module.

[0095] The form parsing module is configured to non-intrusively acquire the UI structure and control semantic information of forms in the target application during the development phase through the front-end integration SDK. This module does not require modification of the host application's source code; instead, it utilizes the SDK to dynamically extract key semantic features of the form interface at runtime, such as hierarchical relationships, control types, label text, placeholders, association descriptions, and dependency logic between controls. This provides a structured and semantically rich input foundation for subsequent mapping generation.

[0096] The semantic mapping generation module is configured to invoke a large language model to automatically generate semantic mapping templates based on the UI structure and control semantic information, exporting and embedding these templates into the host application. These semantic mapping templates are used to establish associations between form items and voice-recognizable keywords to support voice field recognition. This module leverages the powerful natural language understanding and generalization capabilities of the large language model to infer the business semantic role of each form item and generate a set of voice-recognizable keywords covering various commonly used user expressions. This establishes a precise association between form items and voice commands, supporting subsequent voice field recognition functions.

[0097] The runtime switching module is configured to, during the deployment phase, respond to the switch to lightweight mode, load the semantic mapping template, and enable the locally deployed lightweight text parsing and speech recognition models. This module achieves a smooth transition from high-precision modeling during development to efficient execution during deployment, ensuring that the system can run stably even without relying on external large language models, without network connectivity, or in resource-constrained environments, while guaranteeing low latency and high availability.

[0098] The voice parsing module is configured to parse the voice content using a lightweight text parsing model combined with a semantic mapping template upon receiving user voice input, outputting structured field-value pairs. These structured field-value pairs are used to drive at least one downstream application, such as automatic form filling, multimodal confirmation prompts, or process guidance. This module accurately maps the user's natural language voice content to specific form items and extracts the corresponding fill values, generating structured data that can be directly processed by the machine. This data is used to drive downstream applications such as automatic form filling, initiating multimodal confirmation prompts to the user, or triggering subsequent business process guidance, achieving an end-to-end voice-interactive form filling experience.

[0099] In this process, the development and deployment phases are decoupled. The semantic mapping logic is solidified and generated during the development phase, and the deployment phase does not need to rely on a large language model. Furthermore, the entire process does not require modification of the host application's source code.

[0100] Furthermore, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for generating a phased adaptation voice interaction form.

[0101] In summary, the phased adaptation method, system, and medium for generating voice interaction forms provided in this application have the following technical effects.

[0102] This solution significantly reduces manual configuration costs and improves the semantic accuracy of field recognition by non-intrusively acquiring form UI structure and control semantic information using a front-end integration SDK during the development phase, and automatically generating high-coverage semantic mapping templates in conjunction with a large language model. During deployment, it switches to a lightweight operation mode, loading the fixed mapping templates and enabling locally deployed lightweight text parsing and speech recognition models. This eliminates reliance on large language models or network connections, effectively ensuring low latency, high efficiency, and data privacy and security. The entire process decouples intelligent modeling during development from efficient execution during runtime, without requiring modification of the host application's source code. It is compatible with various front-end technology stacks, including Web, H5, mini-programs, and native mobile applications, demonstrating good versatility and engineering feasibility. Furthermore, the system supports context-aware semantic disambiguation, fallback interaction strategies, and a hot-update mechanism for mapping templates, further enhancing the robustness of speech parsing, the continuity of user experience, and the maintainability of the system. This provides a complete, reliable, and scalable technical path for voice-driven intelligent form filling.

[0103] In terms of end-to-end efficiency, during the development phase, semantic mapping rules are automatically generated by calling a large language model, which can reduce the amount of manual configuration work by more than 80% and greatly improve development efficiency. During the deployment phase, lightweight small models and professional ASR models are used for collaborative processing, and the response latency is controlled within 1 second, reducing device memory usage by more than 60% and achieving high-efficiency operation.

[0104] In terms of versatility, this solution adopts a non-intrusive front-end parsing method, combined with a phased model adaptation mechanism, making it compatible with over 95% of front-end application types (including apps, H5, and mini-programs). No code modification to the host application is required, demonstrating strong cross-platform applicability. Regarding accuracy, during the development phase, the large model automatically supplements synonyms and common expression variations (e.g., mapping "medical insurance" to "medical insurance"). Combined with professional ASR noise reduction capabilities and the precise matching mechanism of the small model during the deployment phase, the overall form-filling accuracy is improved to over 95%.

[0105] In terms of integration cost, developers only need to complete three steps—configuring the large model, generating the mapping structure, and switching to the small model mode—with a code volume of no more than 10 lines, significantly lower than the integration complexity of traditional form automation solutions. Regarding flexibility in scenario adaptation, the deployment phase supports flexible switching of ASR models (e.g., using iFlytek domestically and other models overseas). The small model supports local or edge cloud deployment and can adapt to various operating environments such as mobile devices, mini-programs, and the web, meeting diverse business needs. In summary, this solution outperforms existing technologies in terms of efficiency, versatility, accuracy, ease of integration, and deployment flexibility, providing an efficient, reliable, and easily scalable solution for voice-driven intelligent form filling.

[0106] It should be noted that in all specific embodiments of this application, all data processing activities related to user identity or personal characteristics, such as user information, user behavior data, historical data, and location information, will be conducted in accordance with the principles of legality, legitimacy, and necessity. All data collection, use, storage, and processing will be subject to compliance with applicable national and regional laws, regulations, and industry standards, and informed consent from users will be obtained in a clear and explicit manner before processing. For the processing of sensitive personal information, separate consent from users will be obtained through prominent means such as pop-up prompts and independent confirmation pages. If any processing conflicts with laws and regulations, the laws and regulations will prevail, and necessary data processing will only be carried out within the scope permitted by laws and regulations, ensuring that all data-based applications, analyses, and technical implementations are conducted within the scope permitted by laws and regulations.

[0107] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0108] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0109] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0111] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, for example, by optically scanning the paper or other media, then editing, interpreting, or, if necessary, processing it in a suitable manner to obtain the program electronically, and then storing it in computer memory.

[0112] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0113] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0114] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0115] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for generating a phased-adaptive voice interaction form mapping, characterized in that, include: During the development phase, the UI structure and control semantic information of forms in the target application are obtained non-intrusively through the front-end integration SDK; The large language model is invoked to automatically generate a semantic mapping template based on the UI structure and control semantic information, which is then exported and embedded into the host application. The semantic mapping template is used to establish the association between form items and speech-recognizable keywords to support speech field recognition. During the deployment phase, in response to the switch of the runtime configuration to lightweight mode, the semantic mapping template is loaded, and the locally deployed lightweight text parsing model and speech recognition model are enabled. Upon receiving user voice input, the lightweight text parsing model, in conjunction with the semantic mapping template, parses the voice content and outputs structured field-value pairs. These structured field-value pairs are used to drive at least one downstream application, such as automatic form filling, multimodal confirmation prompts, or process guidance. The development phase is decoupled from the deployment phase. The semantic mapping logic is generated and solidified during the development phase, and the deployment phase does not need to rely on a large language model. Furthermore, the entire process does not require modification of the host application's source code.

2. The method for generating a phased, adaptive voice interaction form according to claim 1, characterized in that, The front-end integration SDK is compatible with various front-end technology stacks, including but not limited to Web, H5, mini-programs, and native mobile applications. The non-intrusive acquisition of the UI structure and control semantic information of forms in the target application through the front-end integration SDK includes the following steps: If the target application is a Web, H5, or mini-program application, traverse its Document Object Model (DOM) and extract the form element nodes and their corresponding node attributes. If the target application is a native mobile application, call the accessibility service provided by the operating system to obtain the UI control nodes and their corresponding node attributes in the accessibility tree; Based on the extracted node attributes, the UI structure of the form is constructed, including the parent-child nesting relationship between controls and the spatial layout topology; At the same time, the original semantic clues are obtained from the node attributes, including field label text, placeholder hints, control types, associated description text, and display or selection dependencies between controls; Based on the original semantic clues, the semantic information of the controls corresponding to each form item is determined. The semantic information of the controls includes field labels, control types, and display or selection dependencies between controls.

3. The method for generating a phased, adaptive voice interaction form according to claim 1, characterized in that, The process of calling the large language model and automatically generating a semantic mapping template based on the UI structure and control semantic information includes the following steps: The UI structure and control semantic information are input into the large language model as context. The pre-defined structured prompts guide the large language model to infer the business semantic role of each form item based on the semantic information of the control, and generate a set of voice-recognizable keywords that match each form item; the set of keywords covers multiple common expressions used by users for the same field in natural language. Based on the control type and interaction constraints of each form item, corresponding field parsing rules are generated; these field parsing rules are used to guide the lightweight text parsing model to perform structured parsing of speech recognition results during the online phase. The keyword set and field parsing rules are integrated to form a standardized semantic mapping template, which is then exported in a structured data format.

4. The method for generating a phased-adaptive voice interaction form according to claim 1, characterized in that, The response to switching the runtime configuration to lightweight mode, loading the semantic mapping template, and enabling the locally deployed lightweight text parsing model and speech recognition model includes the following steps: Read the runtime configuration parameters and determine whether the current stage is the deployment stage; During the deployment phase, the semantic mapping template is loaded from the host application's local storage, and its integrity and version validity are verified. Initialize a professional speech recognition model, which is adapted to the form filling scenario and enables endpoint detection and hot word enhancement functions; A quantized and compressed lightweight text parsing model is loaded, which is deployed on a terminal device or edge server for operation in an environment without network connection or where large language models are disabled; wherein, the model compression process includes at least one of quantization, pruning, or knowledge distillation. The speech recognition model and the lightweight text parsing model are bound to the runtime engine of the front-end integration SDK to respond to user voice input and perform field parsing.

5. The method for generating a phased, adaptive voice interaction form according to claim 1, characterized in that, The semantic mapping template supports versioned hot updates after digital signature verification via a secure channel during the deployment phase, specifically including: The update packet of the semantic mapping template is received through a secure channel; Verify the digital signature and version compatibility of the update package; Replace the currently loaded semantic mapping template at runtime without restarting the host application or redeploying the installation package.

6. The method for generating a phased-adaptive voice interaction form according to claim 1, characterized in that, When parsing speech content, the lightweight text parsing model performs context-aware semantic disambiguation based on the currently identified field values ​​and the form business context to distinguish multiple form items with similar semantics.

7. The method for generating a phased, adaptive voice interaction form according to claim 1, characterized in that, When voice parsing fails or no field is matched, a fallback strategy is triggered. The fallback strategy includes prompting the user to repeat, switching to manual input mode, or logging a miss log with the user's authorization, which is used to optimize the semantic mapping template in subsequent development stages.

8. The method for generating a phased-adaptive voice interaction form according to claim 1, characterized in that, While generating semantic mapping templates, the large language model automatically generates a speech-text simulation sample set that covers the diversity of user expressions. The speech-text simulation sample set is used to verify the mapping coverage or fine-tune the lightweight text parsing model.

9. A phased adaptation system for mapping and generating voice-interactive forms, characterized in that, The system includes: The form parsing module is configured to non-intrusively obtain the UI structure and control semantic information of forms in the target application during the development phase through the front-end integration SDK; The mapping generation module is configured to call a large language model to automatically generate a semantic mapping template based on the UI structure and control semantic information, export it and embed it into the host application; the semantic mapping template is used to establish the association between form items and speech-recognizable keywords to support speech field recognition. The running switching module is configured to, during the online phase, in response to the running configuration switching to lightweight mode, load the semantic mapping template and enable the locally deployed lightweight text parsing model and speech recognition model. The voice parsing module is configured to, upon receiving user voice input, use the lightweight text parsing model in conjunction with the semantic mapping template to parse the voice content and output structured field-value pairs; the structured field-value pairs are used to drive at least one downstream application among automatic form filling, multimodal confirmation prompts, or process guidance; The development phase is decoupled from the deployment phase. The semantic mapping logic is generated and solidified during the development phase, and the deployment phase does not need to rely on a large language model. Furthermore, the entire process does not require modification of the host application's source code.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for generating a phased adaptation voice interaction form as described in any one of claims 1 to 8.

Images