A general guide form intelligent filling method and system

Abstract

The application discloses a universal guide type form intelligent filling method and system, and the method comprises the following steps: a user interaction module acquires form application information of a user and sends the form application information to a form intelligent auxiliary engine; the form intelligent auxiliary engine constructs an application form according to the form application information and generates form incomplete element problems according to the application form, and sends the form incomplete element problems to the user interaction module; the user interaction module inquires and acquires input information of the user according to the form incomplete element problems, and sends the input information to an information recognition and marking module; the information recognition and marking module recognizes the input information and generates form element filling information, and sends the form element filling information to the form intelligent auxiliary engine; and the form intelligent auxiliary engine fills the form element filling information into corresponding form elements of the application form. According to the method, the corresponding relationship between the information input by the user and the information required by the form can be automatically recognized, and a specific sequence is not forced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a general wizard-style intelligent form filling method and system. Background Technology

[0002] With the rapid development of information technology, online services have become an indispensable part of modern society, covering various fields from gaming and entertainment, e-commerce to social media and lifestyle tools. These services generally rely on users filling out forms to make requests or issue instructions, serving as a core link in information interaction. Developers of online services meticulously design user interfaces (UI) and user experience (UX) processes to guide users to complete forms efficiently and accurately, thereby improving service quality and user satisfaction.

[0003] However, despite the significant achievements of current information systems in some areas, they still face many challenges, particularly in meeting the needs of diverse user groups. Different user groups, such as the elderly, children, and visually impaired individuals, face specific barriers and limitations when using information systems, and existing systems often lack flexibility and inclusivity, failing to provide input methods and interactive experiences suitable for all users.

[0004] Information systems are often designed with specific business domains and needs in mind, rather than based on users' natural interaction habits. This forces users to follow pre-defined input sequences and logic when filling out forms, resulting in a lack of personalized interaction experiences. Furthermore, the system lacks awareness and understanding of the user's interaction context, making it difficult to provide appropriate guidance and assistance based on the user's actual situation and needs. Summary of the Invention

[0005] The purpose of this invention is to provide a universal wizard-style intelligent form filling method and system, which aims to solve the problem of existing information systems lacking personalized interactive experience.

[0006] In a first aspect, embodiments of the present invention provide a general wizard-style intelligent form filling method, comprising:

[0007] The user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine;

[0008] The form intelligent assistance engine constructs an application form based on the form application information and generates questions about incomplete elements of the form based on the application form, and sends the questions about incomplete elements of the form to the user interaction module.

[0009] The user interaction module asks questions about incomplete elements in the form and obtains the user's input information, then sends the input information to the information recognition and marking module.

[0010] The information recognition and marking module recognizes the input information and generates form element filling information, and sends the form element filling information to the form intelligent assistance engine;

[0011] The intelligent form assistance engine fills in the corresponding form elements of the application form according to the information filled in the form elements.

[0012] Furthermore, before the user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine, it includes:

[0013] The intelligent form assistance engine obtains the service provider's application form template;

[0014] The intelligent form assistance engine decomposes the form elements from the application form template and constructs a corresponding application form template based on the form elements;

[0015] The intelligent form assistance engine saves the application form template to the form template library.

[0016] Furthermore, the form elements include: the information components of the form, the necessity of the information components, the suggested order for filling in the information components, and the method of providing the information components.

[0017] Furthermore, the form intelligent assistance engine constructs an application form based on the form application information and generates incomplete form element issues based on the application form, including:

[0018] The intelligent form assistance engine matches the form application information with all application form templates in the form template library to obtain the corresponding application form template and uses the corresponding application form template as the application form.

[0019] The intelligent form assistance engine obtains user information from the user information database and fills the user information into the form elements corresponding to the form application information to obtain a preliminary filled application form.

[0020] The form intelligent assistance engine obtains the unfilled form elements based on the initially filled application form, and generates incomplete form element questions based on the unfilled form elements.

[0021] Furthermore, the issue of generating incomplete form elements based on unfilled form elements includes:

[0022] The form intelligent auxiliary engine obtains user information from the user information database and behavioral information from the user behavior log database;

[0023] Based on the user information, the unfilled form elements, and the behavior information, generate a question about incomplete form elements.

[0024] Furthermore, the user information includes: the user's basic information, the user's social network, and the services activated by the user.

[0025] Furthermore, the behavioral information includes: the input method used by the user when answering the information items of the form, the content of the information replied by the user when answering the information items of the form, the time when the user answered, and the address when the user answered.

[0026] Furthermore, the information recognition and marking module identifies the input information and generates form element filling information, including:

[0027] The information recognition and tagging module recognizes the input information and extracts the input vector from the input information using the word2vec model;

[0028] The form vector is extracted from the form elements of the application form using the word2vec model;

[0029] Calculate the similarity between the input vector and each form vector;

[0030] If the similarity reaches a predetermined value, the input information corresponding to the input vector will be used as the form element to fill in the information.

[0031] Furthermore, it also includes:

[0032] The form instance library records in-process form information in real time;

[0033] The form instance library records completed form information in real time.

[0034] This invention also provides a general wizard-style intelligent form filling system, comprising:

[0035] The user interaction module is used to obtain the user's form application information and send the form application information to the form intelligent assistance engine;

[0036] The form intelligent assistance engine is used to construct an application form based on the form application information and generate incomplete element questions based on the application form, and send the incomplete element questions to the user interaction module.

[0037] The user interaction module is used to ask questions based on the incomplete elements of the form and obtain the user's input information, and send the input information to the information recognition and marking module;

[0038] The information recognition and marking module is used to recognize the input information and generate form element filling information, and send the form element filling information to the form intelligent assistance engine;

[0039] The form intelligent assistance engine is also used to fill in the corresponding form elements of the application form according to the information filled in by the form elements.

[0040] This invention discloses a general wizard-style intelligent form filling method and system. The method includes: a user interaction module acquiring the user's form application information and sending it to a form intelligent assistance engine; the form intelligent assistance engine constructing an application form based on the application information and generating incomplete form element questions based on the application form, sending these questions to the user interaction module; the user interaction module querying and acquiring the user's input information based on the incomplete form element questions, and sending the input information to an information recognition and marking module; the information recognition and marking module recognizing the input information and generating form element filling information, sending this information to the form intelligent assistance engine; and the form intelligent assistance engine filling the corresponding form elements in the application form based on the form element filling information. This invention, through the form intelligent assistance engine, information recognition and marking module, and user interaction module, can automatically identify the correspondence between the user's entered information and the information required by the form, without forcing a specific order, resulting in a good personalized interactive experience. This invention also provides a general wizard-style intelligent form filling system with the above-mentioned beneficial effects, which will not be elaborated further here. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the general wizard-style intelligent form filling method in this embodiment;

[0043] Figure 2 This is a schematic diagram of the general wizard-style intelligent form filling system of this embodiment;

[0044] Figure 3 This is a sub-flowchart of the general wizard-style intelligent form filling method in this embodiment;

[0045] Figure 4 This is another sub-flowchart of the general wizard-style intelligent form filling method in this embodiment;

[0046] Figure 5A flowchart of a general wizard-style intelligent form filling method according to another embodiment;

[0047] Figure 6 This is a schematic block diagram of a general wizard-style intelligent form filling system according to this embodiment. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0050] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0051] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0052] Please see Figure 1 and Figure 2 This invention provides a universal wizard-style intelligent form filling method, including:

[0053] S101: The user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine;

[0054] In this embodiment, before the user interaction module (i.e., HMI) obtains the user's form application information and sends the form application information to the form intelligent assistance engine (i.e., Form-AI-Co), the following steps are included:

[0055] The intelligent form assistance engine retrieves the application form template from the service provider;

[0056] The intelligent form assistance engine decomposes form elements from the application form template and constructs corresponding application form templates based on these elements.

[0057] The intelligent form assistance engine saves the application form template to the form template library (i.e., Form-Template).

[0058] The system uses an intelligent form-assisted engine to retrieve application form templates from service providers and automatically decomposes the form elements to construct the templates. This process allows the system to quickly generate corresponding forms based on different business needs without hard-coding or manual adjustments. This ability to dynamically generate forms greatly improves the system's flexibility and adaptability. The constructed application form templates are saved to a form template library for easy reuse and management. When the service provider's application form template changes, the system can automatically update or regenerate the template, reducing the possibility of manual intervention and errors.

[0059] The form elements include: the information components of the form, such as a registration form typically including username, password, mobile phone number, etc.; the necessity of the information components, i.e., whether the information component is required; the suggested order for filling in the information components, for example, first fill in the username, then the password, and then the mobile phone number; and the way the information components are provided, such as whether they are provided through text, files, images, or videos.

[0060] In this embodiment, each application form template may consist of multiple records, and each record may contain at least the information in Table 1:

[0061] Table 1

[0062] Serial Number name illustrate Example 1 serial number A unique identifier for a unit of information F001 2 name Briefly describe the name of this information unit. latitude and longitude 3 type Type of this information unit binary array 4 Required Is it required to fill in? whether 5 Example Provide sample data (22.00,101.00) 6 illustrate Explanation of information Detailed geolocation marking

[0063] During the process of the form intelligent assistance engine obtaining the application form template from the service provider and building the application form template, the form intelligent assistance engine will deeply analyze each form element and customize the display and behavior of the form based on its attributes (such as number name, type, required, example and description).

[0064] Each form element is assigned a unique number and an easy-to-understand name. These numbers and names are not only used for internal management and reference, but are also clearly displayed to users on the form interface, helping them quickly locate and understand the information that needs to be filled in.

[0065] Based on the type of form elements (such as text, number, date, selection box, checkbox, etc.), the form intelligent assistance engine automatically selects the appropriate input control and sets the corresponding validation rules. For example, for numeric fields, the form intelligent assistance engine restricts users to only entering numbers; for date fields, it provides a date picker for users to choose from.

[0066] For form elements marked as "required," the form intelligent assistance engine will prompt the user when they attempt to submit the form but have missed these fields, ensuring that the user must fill in all the critical information. This helps improve the completeness and accuracy of the data.

[0067] For certain complex form elements, the form intelligent assistance engine provides sample text or values ​​next to the fields to guide users on how to fill them out correctly. Examples can be common input formats, recommended options, or brief descriptions of the field content, helping to reduce user errors.

[0068] In addition to examples, the form intelligent assistance engine also supports adding detailed explanatory text to each form element. These explanatory texts can further explain the purpose of a field, provide notes on filling it out, or clarify relevant policies. By providing rich explanatory information, the form intelligent assistance engine helps users better understand the form content and make correct selections.

[0069] It should be noted that the form template library is maintained and used by the service provider to maintain various forms and facilitate the service provider in creating various form templates.

[0070] When a user needs to fill out an application form, the user can apply for an application form filling assistance service. Then, the user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine. Then, the wizard-style form intelligent filling system executes step S102.

[0071] S102: The form intelligent assistance engine constructs an application form based on the form application information and generates incomplete form element questions based on the application form, and sends the incomplete form element questions to the user interaction module;

[0072] Specifically, the intelligent form assistance engine constructs an application form based on the application information and generates incomplete elements of the form based on the application form, including:

[0073] The intelligent form assistance engine matches the form application information with all the application form templates in the form template library to obtain the corresponding application form template and use the corresponding application form template as the application form;

[0074] The intelligent form assistance engine retrieves user information from the user information database and fills the user information into the corresponding form elements of the form application information to obtain a preliminary application form.

[0075] The form intelligent assistance engine obtains the unfilled form elements from the initially filled application form and generates incomplete form element questions based on these unfilled elements.

[0076] The automated process greatly simplifies the form creation and completion process. It intelligently matches form application information with templates in the template library, automatically constructing the corresponding application form without requiring manual selection and configuration, significantly improving work efficiency. The intelligent form assistance engine seamlessly integrates with the user information database, automatically filling in the corresponding elements in the form with user information. This function reduces repetitive input, lowers the error rate, and ensures the accuracy and consistency of user data. The intelligent form assistance engine can also identify and highlight unfilled form elements, generating specific questions about incomplete elements. This instant feedback mechanism helps users quickly locate and address missing information, thereby accelerating the completion of the form and reducing rework caused by missing information after submission.

[0077] Specifically, the issue of generating incomplete form elements based on unfilled form elements includes:

[0078] The form intelligent assistance engine obtains user information from the user information database (i.e., UserInfoLib) and behavioral information from the user usage behavior log database (i.e., QALib);

[0079] Generate incomplete form element questions based on user information, unfilled form elements, and behavioral information.

[0080] After obtaining user information from the user information database and behavioral information from the user behavior log database, the form intelligent assistance engine matches the user information with the form elements and fills the matched user information into the matched form elements. Then, the form intelligent assistance engine uses the behavioral information from the user behavior log database to generate questions about incomplete form elements based on the behavioral information.

[0081] In this embodiment, user information includes: basic user information, such as personal name, address, age, etc.; user relationship chain, such as friends, colleagues, etc.; and services activated by the user, such as cloud storage for documents, personal blog, etc. The user's basic information, user relationship chain, and activated services all need to be tagged and categorized so that the system can accurately retrieve them.

[0082] The user information database consists of multiple information records, and each information record includes at least the information in Table 2:

[0083] Table 2

[0084] Serial Number name illustrate Example 1 serial number A unique identifier for an information record F002 2 name Information Unit Name ID card 3 type Type of this information unit document 4 Required Is it required to fill in? whether 5 Example Provide sample data Path or URL: / path / to / sfz 6 illustrate Explanation of information User's ID card image 7 Matching tags A set of tags that can be matched with form information ID, ID card

[0085] In this embodiment, the behavioral information includes: the input method used by the user when answering the information items of the form, such as by voice, keyboard, or file upload; the information content of the user's reply when answering the information items of the form, the time of the user's reply, and the address of the user's reply.

[0086] Specific examples are shown in Table 3:

[0087] Table 3

[0088]

[0089]

[0090] Furthermore, the form intelligent assistance engine uses behavioral information from the behavior log library and generates incomplete form element issues based on this information, including:

[0091] For each user, analyze their behavioral patterns on different forms. Pay special attention to fields that are frequently left blank or entered incorrectly;

[0092] Calculate the incomplete rate (number of times not filled in / total number of visits) and error rate (number of times incorrectly filled in / total number of times filled in) for each field;

[0093] Identify whether users have a specific filling order or skipping pattern. For example, some users may always fill in certain fields first and then skip other fields.

[0094] Based on the incompleteness and error rates, generate incomplete element questions for fields with high incompleteness or error rates. These questions may include:

[0095] "Please make sure you have filled in the [field name], as this is very important for the process."

[0096] "You entered incorrect information in this field before. Please check [field name] again."

[0097] For users with specific filling order or skipping patterns, if it is detected that certain fields that are not normally skipped have been skipped, the following question can be generated:

[0098] "You seem to have skipped [field name]. Is that your intention? If not, please consider filling in this field."

[0099] In some embodiments, when obtaining the information composition items of the user's response form, the user is then guided to continue filling in the form according to the input method used by the user. For example, if the user is accustomed to using voice input to enter their mobile phone number, the user is directly guided to use voice input.

[0100] In this embodiment, after generating the issue of incomplete form elements, the form intelligent assistance engine of this embodiment will call the examples and explanations in the records corresponding to each unfilled form element to provide the user with the information so that the user can fill in the form effectively and accurately. For example, if the user does not fill in the latitude and longitude, the example (22.00, 101.00) and explanation (detailed geographical location marking) can be called according to Table 1 so that the user can fill in the form effectively and accurately.

[0101] Next, the form intelligent assistance engine sends the issue of incomplete form elements to the user interaction module.

[0102] S103: The user interaction module asks questions about incomplete elements of the form and obtains the user's input information, and then sends the input information to the information recognition and marking module;

[0103] In this embodiment, the user interaction module provides multiple information input and output capabilities, including input and output methods such as sound, video, images, files, keyboard, and screen, providing information acquisition and delivery capabilities that meet user habits.

[0104] a) Provides multiple data input methods: HTML-based keyboard input, voice-based input, input based on images, files and videos, and auxiliary input from external systems (including devices and people).

[0105] b) Provides multiple data output methods: including output via screen, speakers, and external system interfaces.

[0106] These combination methods are integrated into a single SDK, supporting localized application integration in browsers and mainstream operating systems including Windows, Android, HamonyOS, and iOS.

[0107] S104: The information recognition and marking module recognizes the input information and generates form element filling information, and sends the form element filling information to the form intelligent auxiliary engine;

[0108] In this embodiment, the Information Identification and Marking Module (IDS) integrates the following capabilities to convert and identify information, making it easier to match it with structured form information.

[0109] Large Language Model Unit (LLM): A unit that matches major mainstream large language models, uses large language models to identify the semantics of user-submitted information, and corresponds it to the specific elements of the form.

[0110] LLM can extract structured information from users' natural language, making it easier to match with the information elements required by the form. For example, if the form requires three elements—time, location, and type of goods—users can provide this information one by one or all at once, in no particular order. The following steps describe this process:

[0111] 1. AI-FORM prompts users: Please provide the time, location, and type of goods.

[0112] 2. The user said: I will pick up a wooden cabinet at Shekou Port at 9:00 AM.

[0113] 3. When requesting LLM using AI-FORM, send the following information:

[0114] Extract the following text by location, time, and goods type, and present the results in JSON format, with the attributes represented by time, location, and goods_type: "Pick up a wooden cabinet at Shekou Wharf at 9:00 AM."

[0115] 4. LLM will return.

[0116] Voice conversion unit (VTT): Converts sound into text to recognize information input by the user via voice.

[0117] Text-to-Speech (TTS) unit: converts text into sound, which can be used to provide feedback on the user's system progress when using voice input.

[0118] Image extraction unit (OCR): Extracts information from images containing text.

[0119] The information identification and tagging module also includes other unlisted capability units for converting and identifying information.

[0120] In this embodiment, the information recognition and marking module identifies input information and generates form element filling information, including:

[0121] The information recognition and tagging module recognizes the input information and extracts the input vector from the input information using the word2vec model;

[0122] The form vector is extracted from the form elements of the application form using the word2vec model;

[0123] Calculate the similarity between the input vector and each form vector;

[0124] If the similarity reaches a predetermined value, the input information corresponding to the input vector will be used as the form element to fill in the information.

[0125] In one embodiment, if the input methods of form elements include not only text information but also other modalities such as images and audio, then multimodal learning techniques are used to extract and combine vector representations of these different modalities.

[0126] Specifically, the relevant image dataset is preprocessed; a basic convolutional neural network model and a feature fusion network model are constructed; the constructed basic convolutional neural network and feature fusion network models are trained using the preprocessed dataset to obtain the model with the corresponding weight parameters, i.e., the trained detection model; and the image data is input into the trained detection model to obtain image features.

[0127] A first text for which text features are to be extracted is determined, and at least one first sliding window and a corresponding sliding step size are used for extracting text features. For each first sliding window, starting from a set initial sliding position, the first sliding window is slid along the arrangement path of the characters constituting the first text with the corresponding sliding step size, and the string in the first sliding window is extracted during the sliding process until the characters constituting the first text are slid over. The extracted string is used as the text features of the first text.

[0128] Acquire audio information; perform preprocessing on the audio information to enhance the performance of the speech signal, resulting in preprocessed audio information; convert the preprocessed audio information from the time domain to the frequency domain using a fast Fourier transform; filter the audio information in the frequency domain using a Mel filter bank with frequencies set based on the language information of the audio information to determine the audio features of the audio information.

[0129] Next, the image features, text features, and audio features are directly concatenated to form a long vector. Then, an attention mechanism is used to learn the weights of the image features, text features, and audio features within the long vector. Finally, a Transformer (deep learning model) is used to map the data of the image features, text features, and audio features into a shared feature space based on the weights of the image features, text features, and audio features within the long vector, thus obtaining the fused form element filling information.

[0130] The system can simultaneously receive and process input information from different modalities, such as text input, voice input, image recognition (e.g., OCR scanning of text or tables in documents), and possible motion capture (e.g., gesture operation to select form items). This multimodal data is integrated through advanced fusion algorithms to fully utilize the advantages of each input method and reduce the errors that may be caused by a single input method.

[0131] S105: The form intelligent auxiliary engine fills in the form elements corresponding to the application form according to the form element information.

[0132] This embodiment also includes:

[0133] The Form-Instance library records in-process form information in real time, so that users can understand the form filling progress and detailed information;

[0134] The form instance library records completed form information in real time, making it convenient for users and service providers to use and review.

[0135] In this embodiment, please refer to Figure 3 Based on the form provided by the service provider, the process for generating an intelligent wizard is as follows: The Form-Tempbte form is decomposed, and the information to be entered is iterated through. Next, it checks if UserInfoLib contains information matching the form elements. If not, the user is guided to enter this information according to the type and user habits. Then, the raw data is converted according to the form requirements (VTT, OCR, etc.). The corresponding probability is then identified through LLM, and it is determined whether the probability meets the threshold. If the probability meets the threshold, the raw information is recorded in UserInfoLib, and the instance is recorded in Form-Instance. If the probability does not meet the threshold, the user is guided to enter this information again according to the type and user habits. If the probability does meet the threshold, the instance is recorded in Form-Instance. Next, it is determined whether the information for the next form has been obtained. If the information for the next form has been obtained, it checks if UserInfoLib contains information matching the form elements. If the information for the next form has not been obtained, the operation ends.

[0136] In this embodiment, please refer to Figure 4 The process, which is based on the continuous evolution of UserInfoLib, Form-Template, and QALib, is as follows: extract data from UserInfoLib, Form-Template, and QALib; then calculate the frequency of semantic keyword combinations of prompt information, user information, and form information; and then update the Usetnlolib matching tag set according to the combination frequency.

[0137] In this embodiment, the question (the element information required by the form), the answer (the information submitted by the user), and the answer provision method (the input method used by the user) will be continuously accumulated in UserInfoLib, Form-Template, and QALib. The matching frequency of their information can be statistically analyzed, and the correspondence with high matching frequency will be recorded, while the correspondence with high matching frequency will be deleted.

[0138] User-submitted forms are processed by the service provider. The service marks the responses to determine their suitability and verifies the correspondence of each element. A correspondence means the question and answer match; a mismatch means they don't. By statistically analyzing the frequency of matches for a specific set of questions and answers, high-frequency matches are recorded, while low-frequency matches are deleted. The original QALib records look something like this:

[0139] Form information elements User Answer No match address Shenzhen Nanshan Deep Bay No. 1 yes age girl no

[0140] In one embodiment, see Figure 5 A common wizard-style intelligent form completion method includes: the service provider creates a template for the required form content in the intelligent form system. AI-FORM automatically breaks down the form elements; the intelligent form creates a form instance environment for the user and generates an auxiliary wizard to guide the completion of wizard questions; the user can submit the required elements of the form via voice, uploading files, filling out the form, specifying a third party, or system assistance. The intelligent form automatically maps to the user's form instance environment and prompts the user on the progress of completing the form until the user completes all the required information input; the user views the form and confirms submission. The intelligent form records the form and submits the completed form to the service provider; after the service provider approves the form, the intelligent form synchronizes the approval to the user.

[0141] The intelligent form system records and saves the entire process of a user filling out a form, including the information entered, timestamps, and operation logs. This traceability facilitates subsequent data analysis, auditing, and compliance checks, ensuring data integrity and security.

[0142] It should be noted that the service provider here can be a person or various information service systems.

[0143] This embodiment supports richer information collection methods and caters to a wider range of users through the aforementioned approach. Based on user-built information databases and context, forms can be completed more conveniently and accurately. It automatically identifies the correspondence between user-entered information and the information required by the form, without forcing a specific order. It can combine user information and historical form-filling behavior to create a self-optimizing wizard, continuously improving the user experience. It can function as a standalone product, connecting to external information service systems based on forms, accelerating system integration.

[0144] Please see Figure 6 This invention provides a universal wizard-style intelligent form filling system 100, comprising:

[0145] User interaction module 101 is used to obtain the user's form application information and send the form application information to the form intelligent assistance engine;

[0146] The form intelligent assistance engine 102 is used to construct an application form based on the form application information and generate incomplete form element questions based on the application form, and send the incomplete form element questions to the user interaction module 101.

[0147] User interaction module 101 is used to ask questions about incomplete elements of the form and obtain the user's input information, and send the input information to the information recognition and marking module;

[0148] The information recognition and marking module 103 is used to recognize the input information and generate form element filling information, and send the form element filling information to the form intelligent auxiliary engine 102.

[0149] The form intelligent assistance engine 102 is also used to fill in the form elements corresponding to the application form according to the form element information.

[0150] Furthermore, it also includes:

[0151] The form intelligent auxiliary engine 102 is also used to obtain the service provider's application form template;

[0152] The form intelligent auxiliary engine 102 is also used to decompose the form elements from the application form template and construct a corresponding application form template based on the form elements;

[0153] The form intelligent auxiliary engine 102 is also used to save the application form template to the form template library 104.

[0154] Furthermore, the form elements include: the information components of the form, the necessity of the information components, the suggested order for filling in the information components, and the method of providing the information components.

[0155] Furthermore, it also includes:

[0156] The form intelligent auxiliary engine 102 is also used to match the form application information with all application form templates in the form template library 104 to obtain the corresponding application form template and use the corresponding application form template as the application form;

[0157] The form intelligent auxiliary engine 102 is also used to obtain user information in the user information database, fill the user information into the form elements corresponding to the form application information, and obtain a preliminary filled application form;

[0158] The form intelligent assistance engine 102 is also used to obtain unfilled form elements based on the initially filled application form, and generate incomplete form element questions based on the unfilled form elements.

[0159] Furthermore, it also includes:

[0160] The form intelligent auxiliary engine 102 is also used to obtain user information in the user information database 105 and behavior information in the user behavior log database 106;

[0161] Based on the user information, the unfilled form elements, and the behavior information, generate a question about incomplete form elements.

[0162] Furthermore, the user information includes: the user's basic information, the user's social network, and the services activated by the user.

[0163] Furthermore, the behavioral information includes: the input method used by the user when answering the information items of the form, the content of the information replied by the user when answering the information items of the form, the time when the user answered, and the address when the user answered.

[0164] Furthermore, it also includes:

[0165] The information recognition and labeling module 103 is used to recognize the input information and extract the input vector from the input information through the word2vec model;

[0166] The form vector is extracted from the form elements of the application form using the word2vec model;

[0167] Calculate the similarity between the input vector and each form vector;

[0168] If the similarity reaches a predetermined value, the input information corresponding to the input vector will be used as the form element to fill in the information.

[0169] Furthermore, it also includes:

[0170] Form instance library 107 is used to record in-process form information in real time;

[0171] The form instance library 107 is also used to record completed form information in real time.

[0172] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0173] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

[0174] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusivity.

[0175] The term "comprises" implies that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A universal wizard-style intelligent form filling method, characterized in that, include: The user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine; The form intelligent assistance engine constructs an application form based on the form application information and generates questions about incomplete elements of the form based on the application form, and sends the questions about incomplete elements of the form to the user interaction module. The user interaction module asks questions about incomplete elements in the form and obtains the user's input information, then sends the input information to the information recognition and marking module. The information recognition and marking module recognizes the input information and generates form element filling information, and sends the form element filling information to the form intelligent assistance engine; The intelligent form assistance engine fills in the corresponding form elements of the application form according to the information filled in the form elements. The intelligent form assistance engine constructs an application form based on the application information and generates incomplete form elements based on the application form, including: The intelligent form assistance engine matches the form application information with all application form templates in the form template library to obtain the corresponding application form template and uses the corresponding application form template as the application form. The intelligent form assistance engine obtains user information from the user information database and fills the user information into the form elements corresponding to the form application information to obtain a preliminary filled application form. The form intelligent assistance engine obtains the unfilled form elements based on the initially filled application form, and generates incomplete form element questions based on the unfilled form elements; Construct a basic convolutional neural network model and a feature fusion network model; train the constructed basic convolutional neural network and feature fusion network models using a preprocessed dataset to obtain a trained detection model; input image data into the trained detection model to obtain image features; determine the first text from which text features are to be extracted, and at least one first sliding window and corresponding sliding step size for extracting text features; for each first sliding window, starting from a set initial sliding position, slide the first sliding window along the arrangement path of the characters constituting the first text with the corresponding sliding step size, and extract the string within the first sliding window during the sliding process until all characters constituting the first text have been slid over; use the extracted string as the text features of the first text; and process the audio... The information is processed through preprocessing to enhance the performance of the speech signal, resulting in preprocessed audio information. The preprocessed audio information is then converted from the time domain to the frequency domain using a Fast Fourier Transform. The frequency domain audio information is then filtered using a Mel filter bank with frequencies set based on the language information of the audio information to determine its audio features. Next, image features, text features, and audio features are directly concatenated to form a long vector. An attention mechanism is then used to learn the weights of these features within the long vector. Finally, a deep learning model is used to map the image, text, and audio features to a shared feature space based on their weights within the long vector, resulting in fused form element information. The issue of generating incomplete form elements based on unfilled form elements includes: The form intelligent auxiliary engine obtains user information from the user information database and behavioral information from the user behavior log database; Based on the user information, the unfilled form elements, and the behavior information, generate a form incomplete element question; The information recognition and marking module recognizes the input information and generates form element filling information, including: The information recognition and tagging module recognizes the input information and extracts the input vector from the input information using the word2vec model; The form vector is extracted from the form elements of the application form using the word2vec model; Calculate the similarity between the input vector and each form vector; If the similarity reaches a predetermined value, the input information corresponding to the input vector will be used as the form element to fill in the information. Based on the forms provided by the service provider, the process for generating an intelligent wizard is as follows: Decompose the forms in the form template library and iterate through the information to be entered; then check if there is information in the user information database that matches the form elements; if not, guide the user to enter this information according to the type and user habits; then transform the raw data according to the form requirements; then identify the corresponding probability through a large language model and determine if the probability meets the threshold; if the probability meets the threshold, record the raw information in the user information database and record the instance in the form instance database; if the probability does not meet the threshold, continue to guide the user to enter this information according to the type and user habits; if there is, record the instance in the form instance database; then determine if the information for the next form has been obtained; if the information for the next form has been obtained, check if there is information in the user information database that matches the form elements; if the information for the next form has not been obtained, end the operation. The continuously evolving process based on the user information database, form template database, and user behavior log database is as follows: extract data from the user information database, form template database, and user behavior log database; calculate the frequency of semantic keyword combinations of prompt information, user information, and form information; and update the Usetnlolib matching tag set according to the combination frequency.

2. The general wizard-style intelligent form filling method according to claim 1, characterized in that, Before the user interaction module obtains the user's form application information and sends the form application information to the form intelligent assistance engine, it includes the following: The intelligent form assistance engine obtains the service provider's application form template; The intelligent form assistance engine decomposes the form elements from the application form template and constructs a corresponding application form template based on the form elements; The intelligent form assistance engine saves the application form template to the form template library.

3. The general wizard-style intelligent form filling method according to claim 2, characterized in that, The form elements include: the information components of the form, the necessity of the information components, the suggested order for filling in the information components, and the method of providing the information components.

4. The general wizard-style intelligent form filling method according to claim 1, characterized in that, The user information includes: the user's basic information, the user's social network, and the services the user has activated.

5. The general wizard-style intelligent form filling method according to claim 1, characterized in that, The behavioral information includes: the input method used by the user when answering the information items of the form, the content of the information replied by the user when answering the information items of the form, the time when the user answered, and the address when the user answered.

6. The general wizard-style intelligent form filling method according to claim 1, characterized in that, Also includes: The form instance library records in-process form information in real time; The form instance library records completed form information in real time.

7. A universal wizard-driven intelligent form filling system, characterized in that, include: The user interaction module is used to obtain the user's form application information and send the form application information to the form intelligent assistance engine; The form intelligent assistance engine is used to construct an application form based on the form application information and generate questions about incomplete elements of the form based on the application form, and send the questions about incomplete elements of the form to the user interaction module. The user interaction module is used to ask questions based on the incomplete elements of the form and obtain the user's input information, and send the input information to the information recognition and marking module; The information recognition and marking module is used to recognize the input information and generate form element filling information, and send the form element filling information to the form intelligent assistance engine; The form intelligent assistance engine is also used to fill in the corresponding form elements of the application form according to the information filled in the form elements; Also includes: The form intelligent assistance engine is also used to match the form application information with all application form templates in the form template library 104 to obtain the corresponding application form template and use the corresponding application form template as the application form; The form intelligent assistance engine is also used to obtain user information from the user information database, fill the user information into the form elements corresponding to the form application information, and obtain a preliminary filled application form. The form intelligent assistance engine is also used to obtain unfilled form elements based on the initially filled application form, and to generate incomplete form element questions based on the unfilled form elements. The information recognition and labeling module is further used to construct a basic convolutional neural network model and a feature fusion network model; train the constructed basic convolutional neural network and feature fusion network model using a preprocessed dataset to obtain a trained detection model; input image data into the trained detection model to obtain image features; determine the first text to be extracted, and at least one first sliding window and corresponding sliding step size for extracting text features; for each first sliding window, starting from a set initial sliding position, slide the first sliding window along the arrangement path of the characters constituting the first text with the corresponding sliding step size, and extract the string within the first sliding window during the sliding process until all the characters constituting the first text have been slid over; and use the extracted string as the text of the first text. This feature involves: performing preprocessing on the audio information to enhance the performance of the speech signal, obtaining preprocessed audio information; converting the preprocessed audio information from the time domain to the frequency domain using a Fast Fourier Transform; filtering the frequency domain audio information through a Mel filter bank with frequencies set based on the language information of the audio information to determine the audio features; then directly concatenating the image features, text features, and audio features to form a long vector; then using an attention mechanism to learn the weights of the image features, text features, and audio features within the long vector; finally, using a deep learning model and based on the weights of the image features, text features, and audio features within the long vector, mapping the data of the image features, text features, and audio features to a shared feature space to obtain the fused form element filling information. The form intelligent assistance engine is also used to obtain user information in the user information database and behavioral information in the user behavior log database; Based on the user information, the unfilled form elements, and the behavior information, generate a form incomplete element question; The information recognition and labeling module is used to recognize the input information and extract the input vector from the input information using the word2vec model; The form vector is extracted from the form elements of the application form using the word2vec model; Calculate the similarity between the input vector and each form vector; If the similarity reaches a predetermined value, the input information corresponding to the input vector will be used as the form element to fill in the information. Based on the forms provided by the service provider, the process for generating an intelligent wizard is as follows: Decompose the forms in the form template library and iterate through the information to be entered; then check if there is information in the user information database that matches the form elements; if not, guide the user to enter this information according to the type and user habits; then transform the raw data according to the form requirements; then identify the corresponding probability through a large language model and determine if the probability meets the threshold; if the probability meets the threshold, record the raw information in the user information database and record the instance in the form instance database; if the probability does not meet the threshold, continue to guide the user to enter this information according to the type and user habits; if there is, record the instance in the form instance database; then determine if the information for the next form has been obtained; if the information for the next form has been obtained, check if there is information in the user information database that matches the form elements; if the information for the next form has not been obtained, end the operation. The continuously evolving process based on the user information database, form template database, and user behavior log database is as follows: extract data from the user information database, form template database, and user behavior log database; calculate the frequency of semantic keyword combinations of prompt information, user information, and form information; and update the Usetnlolib matching tag set according to the combination frequency.

Images