A questionnaire automatic generation and optimization method based on a large language model

Abstract

The application belongs to the technical field of natural language processing and human-computer interaction, and discloses a questionnaire automatic generation and optimization method based on a large language model, which comprises the following steps: inputting a natural language requirement or uploading a research document, calling an LLM after file preprocessing and context construction; directly generating a single round for a short questionnaire, and entering a staged pipeline for a long questionnaire; generating a structured questionnaire after high-fault-tolerance analysis of the LLM output YAML, storing the questionnaire in a database and creating a questionnaire version; and viewing differences and selectively merging in the version comparison interface question by question. The application adopts the above method, adopts a staged pipeline for a long research document, breaks through the length bottleneck of single output, and stably generates a long questionnaire with hundreds of questions; adopts a two-step algorithm of semantic detection and deterministic expansion for a loop question, automatically expands a placeholder question into a puttable question; and adopts field-level difference detection and selective merging of question number anchor point alignment for a multi-version questionnaire, so that the user can accurately control each adoption.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and human-computer interaction, and in particular to a method for automatic generation and optimization of questionnaires based on a large language model. Background Technology

[0002] Online questionnaires are a core tool for market research, user feedback and product evaluation. Traditional questionnaire creation relies on manual writing of each question, which has the following problems: (1) From customer research data to structured questionnaires, it is necessary to determine the question type, options, jump logic and display logic for each question, which is time-consuming and prone to errors; (2) When editing questionnaires for long research documents, the number of questions can reach more than 100, which is a huge workload.

[0003] In recent years, Large Language Model (LLM) technology has developed rapidly, possessing powerful natural language understanding and generation capabilities, and gradually supporting multimodal input—capable of simultaneously processing information from multiple modalities such as text, images, tables, and documents, and possessing capabilities such as long context understanding, logical reasoning, and structured output (such as YAML and JSON). This makes it possible for AI to assist in generating structured content through natural language dialogue or uploaded documents, providing a new technical means for the automatic generation of questionnaires.

[0004] However, applying large language models directly to questionnaire generation still faces many challenges: the model's single output length is limited, and it is prone to content truncation and formatting errors when dealing with long documents; it lacks sufficient understanding of business semantics such as loop questions and jump / display logic unique to market research; it lacks fine comparison and merging methods for multiple versions generated by multiple rounds of interaction; and its output structured text format is often non-standard, with poor parsing fault tolerance.

[0005] To truly apply the capabilities of large language models to questionnaire creation scenarios, this invention provides a unified recognition and processing capability for various survey document formats as its technological foundation: the system can recognize and parse mainstream document formats such as Word, Excel, and PDF—for Office documents (Word / Excel), it uses document conversion technology to convert them into Markdown text or PDF; for PDF documents, it extracts the text or directly inputs it into the large language model in a multimodal manner (such as converting the page into an image and then encoding it with base64); and the parsed content is then segmented and summarized to construct a dialogue context for the large language model to reference when generating questionnaires. Based on this, this invention further proposes methods such as phased generation, automatic expansion of cyclical questions, and version difference comparison to address the aforementioned challenges. Summary of the Invention

[0006] The purpose of this invention is to provide a method for automatic generation and optimization of questionnaires based on a large language model. For long survey documents, a phased pipeline of "outline extraction - segmentation generation - deterministic assembly" is adopted to break through the bottleneck of single output length and stably generate questionnaires with hundreds of questions. For cyclical questions, a two-step algorithm of "semantic detection - deterministic expansion" is adopted to automatically expand placeholder questions into deployable questions. For multiple versions of questionnaires, field-level difference detection and selective merging with question number anchor alignment are adopted to allow users to accurately control each adoption.

[0007] To achieve the above objectives, this invention provides a method for automatic questionnaire generation and optimization based on a large language model, which consists of five modules: questionnaire generation pipeline, loop question processing, version management, fault tolerance parsing, and context management. First, users input their natural language requirements or upload research documents through the dialogue interface. After file preprocessing and context building, the system calls the Large Language Model (LLM). Secondly, short questionnaires are generated in a single round, while long questionnaires are processed in a phased pipeline. Then, the YAML output by LLM is parsed with high fault tolerance to generate a structured questionnaire, which is then stored in the database and a questionnaire version is created. Finally, users can view the differences question by question on the version comparison interface and selectively merge them before finally publishing them to the questionnaire distribution platform.

[0008] Preferably, for long survey documents, a phased questionnaire generation method is adopted to avoid failure of a single LLM call due to excessively long output. Specifically, this includes: (1) Stage 1 outline extraction; (2) Stage 2 segmented generation; (3) Stage 3 Deterministic assembly.

[0009] The preferred process for extracting the outline in Stage 1 is as follows: First, a deterministic algorithm is used to perform structured summary compression on the document—regular expressions are used to identify chapter titles, question number lines, and structural marker words, redundant text is removed, and a compact outline text is generated; Then, call LLM to output a JSON outline, which contains several sections. Each section records the section identifier, name, and set of question numbers involved.

[0010] Preferably, the specific process of generating stage 2 segments is as follows: For each section in the outline, the corresponding source text is precisely sliced ​​from the original document based on its question number set, and the LLM is called separately to generate the questions for that section. The results of each section are persisted independently as a segment file. At the same time, a list file is maintained to record the generation progress of each section, and it supports resuming the query after interruption.

[0011] Preferably, the specific process of stage 3 deterministic assembly is as follows: First, the questions in each section are merged according to the outline order; then, a deterministic cyclical question unfolding is performed to generate a complete questionnaire; the assembly stage does not rely on LLM and the results are reproducible.

[0012] The preferred method for detecting and deterministically expanding cyclic problems is as follows: Step 1, Semantic Detection: First, use multiple sets of Chinese semantic regular expressions to identify loop feature words—strong loop markers, context loop markers, and placeholder markers; Then, establish a mapping between question number → question and multiple choice → option, and locate the preceding multiple choice that the current loop question depends on by reverse lookup; Finally, use regular expressions to exclude expanded questions and output expansion suggestions for the source question → question to be expanded → source option list; Step 2, Deterministic Expansion: For each question to be expanded, perform deterministic copying and rewriting based on the number of options in the source question: replace placeholders, rewrite question numbers, and rewrite question numbers referenced in the jump logic.

[0013] Preferably, the implementation process for questionnaire version management and multi-dimensional difference comparison is as follows: (1) Data model: A dual-track versioning system is adopted - each dialogue session has one and only one version marked as the final version, and multiple dialogue versions, which form a version chain through self-referencing fields; (2) Difference detection algorithm: Establish a mapping between left and right versions of questions with question number as anchor point, take the union of question numbers, and sort them naturally according to "basic question number + suffix". Classify questions into four categories: same / modified / added / deleted. For questions with the same question number, compare the fields of question number, question stem, question type, instructions, options, matrix rows / columns, jump logic, display logic, verification rules, and option attributes layer by layer. For differences in question stem, use the longest common prefix / suffix algorithm to locate the changed segments and perform character-level addition, deletion, and highlighting. (3) Selective merging: Supports single question replacement / insertion, as well as batch replacement / insertion after selecting multiple questions, only writing back the questions selected by the user to the final version, avoiding full overwrite and loss of manual editing; (4) Performance optimization: The question comparison row uses a memoized component and stable callback to ensure smooth rendering of the list of hundreds of questions.

[0014] The preferred, highly fault-tolerant YAML parsing process is as follows: First, code block extraction: use a state machine to parse Markdown code blocks character by character, and handle nested backticks and consecutive code blocks; Then, multi-level error recovery: first, preprocess and repair invalid escape sequences within double quotes; then detect indentation anomalies and complete the indentation for retry; finally, extract the line number of the error from the error message using regular expressions, delete the line, and retry. Finally, dual-format normalization: It supports both Chinese field list format and English field dictionary format, and outputs a standardized structure for front-end consumption.

[0015] Preferably, the specific process of dialogue context management is as follows: First, the uploaded long document is divided into sliding blocks with a fixed length and overlapping window, and stored as multiple block context records. Then, LLM is called to generate summary context. Then, the YAML of the questionnaire generated in each round is persisted as a YAML context and injected with the LLM hints for the next round, realizing cross-round memorization of questionnaire content and supporting iterative dialogues that can be modified based on the current questionnaire; Finally, a greedy truncation algorithm is used to truncate historical messages based on the token budget, always retaining system messages and the latest user messages.

[0016] Therefore, the present invention employs the above-mentioned method for automatic questionnaire generation and optimization based on a large language model, and the beneficial effects are as follows: (1) Breaking through the bottleneck of long questionnaire generation: Through the phased pipeline of “outline extraction - segmented generation - deterministic assembly”, the single ultra-long output is broken down into multiple controllable short outputs, which can stably generate long questionnaires of hundreds of questions. Moreover, the products of each stage are traceable and support breakpoint continuation, which is significantly better than the existing single generation scheme.

[0017] (2) Automatic expansion of loop questions: The first two-step method of "semantic detection + deterministic expansion" is used to automatically expand the loop question placeholders output by LLM into specific deployable questions. The expansion rules are deterministic and reproducible, eliminating the need for manual splitting, which greatly reduces the production cost of market research questionnaires.

[0018] (3) Version differences are precise and controllable: Field-level difference detection and character-level highlighting with question number anchor point alignment are adopted. Combined with single question / batch selective merging, users can accurately grasp and control each round of AI modification, avoid loss of manual editing caused by full coverage, and improve the efficiency of human-computer collaboration and the credibility of results.

[0019] (4) High availability of generated results: The state machine code block extraction combined with three levels of error recovery, including escape repair, indentation repair, and line deletion, significantly improves the parsing success rate of non-standard LLM output.

[0020] (5) Cross-round memory and controllable cost: YAML context persistence enables iterative modification of questionnaire content, and Token budget greedy truncation controls call cost while retaining key context.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] Figure 1This is the overall flowchart of a questionnaire automatic generation and optimization method based on a large language model according to the present invention; Figure 2 This is a flowchart of the phased questionnaire generation process of the present invention; Figure 3 This is a flowchart of the loop problem detection and expansion process of this invention; Figure 4 This is a flowchart of the version difference comparison and selective merging process of this invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] like Figure 1 As shown, the present invention provides an automatic questionnaire generation and optimization method based on a large language model, which consists of five major modules: questionnaire generation pipeline, loop question processing, version management, fault tolerance parsing, and context management.

[0025] First, users input their natural language requirements or upload research documents (Word / Excel / PDF) through a dialog interface. After file preprocessing and context building, the system calls the Large Language Model (LLM).

[0026] Secondly, short questionnaires are generated in a single round, while long questionnaires are processed in a phased pipeline. Then, the YAML output by LLM is parsed with high fault tolerance to generate a structured questionnaire, which is then stored in the database and a questionnaire version is created. Finally, users can view the differences question by question on the version comparison interface and selectively merge them before finally publishing them to the questionnaire distribution platform.

[0027] like Figure 2 As shown, for long survey documents, a phased questionnaire generation method is adopted to avoid failure of a single LLM call due to excessively long output. The specific process is as follows: (1) Stage 1 Outline Extraction: First, a deterministic algorithm is used to perform structured summary compression on the document—regular expressions are used to identify chapter titles (such as “Part X”), question number lines (such as “Q1.”, “1.1”), and structural marker words (insertion / variable / jump / display / random), and redundant text is removed to generate a compact outline text; then, LLM is called to output a JSON outline, which contains several segments, each of which records the segment identifier, name and the set of question numbers involved.

[0028] (2) Stage 2 segmented generation: For each segment in the outline, the corresponding source text is precisely sliced ​​from the original document according to its question number set, and LLM is called separately to generate the questions for that segment (the number of questions in a single segment is set to an upper limit, such as 18 questions). The results of each segment are persisted independently as a segment file; at the same time, a list file is maintained to record the generation progress of each segment, and it supports resuming the query after interruption.

[0029] (3) Stage 3 Deterministic Assembly: Merge the questions of each section according to the outline, and then perform deterministic cyclic question expansion to generate a complete questionnaire; among them, the assembly stage does not depend on LLM and the results are reproducible.

[0030] The phased questionnaire generation process breaks down a single, extremely long output into multiple controllable, short outputs, making it possible to generate questionnaires with hundreds of questions, and ensuring that the output of each phase is traceable and recoverable.

[0031] like Figure 3 As shown, the survey contains recurring questions (such as "Answer separately for each brand you selected in Q3"). LLM typically outputs these as placeholders and processes them in two steps: Step 1, Semantic Detection: First, use multiple sets of Chinese semantic regular expressions to identify loop feature words—strong loop markers ("selected brand", "each brand you selected"), context loop markers ("each brand", "one by one", "loop question"), and placeholder markers ("insert", "substitute", "this brand"). Then, establish a mapping between question number → question and multiple choice → option, and locate the preceding multiple choice questions that the current loop question depends on by reverse lookup. Finally, use the expanded question regular expression ("question number_number") to exclude expanded questions and output expansion suggestions for "source question → question to be expanded → source option list".

[0032] Step 2, Deterministic Expansion: For each question to be expanded, deterministically copy and rewrite it according to the number of options in the source question: replace placeholders (supports various brackets such as 【】, [], (), () and more than 20 generalized tags such as "this brand" and "this category"), rewrite the question number (add "_1" or "_2" suffixes), and rewrite the question number referenced in the jump logic (such as Q.3→Q.3_1).

[0033] The above process automatically converts semantic-level loop descriptions into specific, deployable questions without manual splitting, and the expansion rules are completely deterministic and reproducible.

[0034] like Figure 4 As shown, the implementation process of questionnaire version management and multi-dimensional difference comparison is as follows: (1) Data model: adopts a "dual-track versioning system" - each dialogue session has one and only one version marked as the final version (the basic version confirmed by the user), and multiple dialogue versions (generated each time LLM), and the versions form a version chain through self-referencing fields.

[0035] (2) Difference detection algorithm: Establish a mapping between left and right versions of questions with question number as anchor point, take the union of question numbers, and sort them naturally by "basic question number + suffix" (support splitting question number such as "3-1" and "3a"). Classify questions into four categories: same / modified / added / deleted. For questions with the same question number, compare the question number, question stem, question type, description, options (compare number and text item by item), matrix rows / columns, jump logic, display logic, verification rules, option attributes and other fields layer by layer. For differences in question stem, use the longest common prefix / suffix algorithm to locate the changed segments and add, delete and highlight at the character level.

[0036] (3) Selective merging: Supports single question replacement / insertion, as well as batch replacement / insertion after selecting multiple questions. Only the questions selected by the user are written back to the final version to avoid full overwrite and loss of manual editing.

[0037] (4) Performance optimization: The question comparison row uses a memoized component and stable callback to ensure smooth rendering of the list of hundreds of questions.

[0038] The specific process of high-fault-tolerant YAML parsing is as follows: First, code block extraction: Markdown code blocks are parsed character by character using a state machine (text state / code block start state / code block end state), which is more robust than regular expressions in handling nested backticks and consecutive code blocks.

[0039] Then, multi-level error recovery: first, preprocess and repair invalid escape sequences within double quotes (such as \E, \S, double escape backslashes); then detect indentation anomalies and complete the indentation for retry; finally, extract the line number of the error from the error message using regular expressions, delete the line, and retry.

[0040] Finally, dual-format normalization: It supports both Chinese field list format and English field dictionary format, and outputs a standardized structure for front-end consumption.

[0041] The specific process of dialogue context management is as follows: First, the uploaded long document is divided into sliding blocks with a fixed length and overlapping window (prioritizing breaking at sentence boundaries), saved as multiple block context records, and LLM is called to generate summary context; Then, the YAML of the questionnaire generated in each round is persisted as a YAML context and injected with the LLM hints for the next round, realizing cross-round memorization of questionnaire content and supporting iterative dialogue of "modifying based on the current questionnaire"; Finally, a greedy truncation algorithm is used to truncate historical messages based on the token budget, always retaining system messages and the latest user messages.

[0042] Therefore, this invention adopts the above-mentioned method for automatic questionnaire generation and optimization based on a large language model. For long survey documents, it uses a phased pipeline of "outline extraction - segmentation generation - deterministic assembly" to break through the bottleneck of single output length and stably generate questionnaires with hundreds of questions. For cyclical questions, it uses a two-step algorithm of "semantic detection - deterministic expansion" to automatically expand placeholder questions into deployable questions. For multiple versions of questionnaires, it uses field-level difference detection and selective merging with question number anchor alignment to allow users to accurately control each adoption.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for automatic questionnaire generation and optimization based on a large language model, characterized in that: It consists of five major modules: questionnaire generation pipeline, loop question processing, version management, fault tolerance parsing, and context management. First, users input their natural language requirements or upload research documents through the dialogue interface. After file preprocessing and context building, the system calls the Large Language Model (LLM). Secondly, short questionnaires are generated in a single round, while long questionnaires are processed in a phased pipeline. Then, the YAML output by LLM is parsed with high fault tolerance to generate a structured questionnaire, which is then stored in the database and a questionnaire version is created. Finally, users can view the differences question by question on the version comparison interface and selectively merge them before finally publishing them to the questionnaire distribution platform.

2. The method for automatic questionnaire generation and optimization based on a large language model according to claim 1, characterized in that, For lengthy survey documents, a phased questionnaire generation approach is adopted to avoid failure of a single LLM call due to excessively long output. Specifically, this includes: (1) Stage 1 outline extraction; (2) Stage 2 segmented generation; (3) Stage 3 Deterministic assembly.

3. The method for automatic questionnaire generation and optimization based on a large language model according to claim 2, characterized in that, The specific process of extracting the outline for Phase 1 is as follows: First, a deterministic algorithm is used to perform structured summary compression on the document—regular expressions are used to identify chapter titles, question number lines, and structural marker words, redundant text is removed, and a compact outline text is generated; Then, call LLM to output a JSON outline, which contains several sections. Each section records the section identifier, name, and set of question numbers involved.

4. The method for automatic questionnaire generation and optimization based on a large language model according to claim 2, characterized in that, The specific process of segment generation in Phase 2 is as follows: For each section in the outline, the corresponding source text is precisely sliced ​​from the original document based on its question number set, and the LLM is called separately to generate the questions for that section. The results of each section are persisted independently as a segment file. At the same time, a list file is maintained to record the generation progress of each section, and it supports resuming the query after interruption.

5. The method for automatic questionnaire generation and optimization based on a large language model according to claim 2, characterized in that, The specific process of stage 3 deterministic assembly is as follows: First, the questions in each section are merged according to the outline order; then, a deterministic cyclical question unfolding is performed to generate a complete questionnaire; the assembly stage does not rely on LLM and the results are reproducible.

6. The method for automatic questionnaire generation and optimization based on a large language model according to claim 1, characterized in that, The specific process of loop problem detection and deterministic expansion is as follows: Step 1, Semantic Detection: First, use multiple sets of Chinese semantic regular expressions to identify loop feature words—strong loop markers, context loop markers, and placeholder markers; Then, establish a mapping between question number → question and multiple choice → option, and locate the preceding multiple choice that the current loop question depends on by reverse lookup; Finally, use regular expressions to exclude expanded questions and output expansion suggestions for the source question → question to be expanded → source option list; Step 2, Deterministic Expansion: For each question to be expanded, perform deterministic copying and rewriting based on the number of options in the source question: replace placeholders, rewrite question numbers, and rewrite question numbers referenced in the jump logic.

7. The method for automatic questionnaire generation and optimization based on a large language model according to claim 1, characterized in that, The implementation process of questionnaire version management and multi-dimensional difference comparison is as follows: (1) Data model: A dual-track versioning system is adopted - each dialogue session has one and only one version marked as the final version, and multiple dialogue versions, which form a version chain through self-referencing fields; (2) Difference detection algorithm: Establish a mapping between left and right versions of questions with question number as anchor point, take the union of question numbers, and sort them naturally according to "basic question number + suffix". Classify questions into four categories: same / modified / added / deleted. For questions with the same question number, compare the fields of question number, question stem, question type, instructions, options, matrix rows / columns, jump logic, display logic, verification rules, and option attributes layer by layer. For differences in question stem, use the longest common prefix / suffix algorithm to locate the changed segments and perform character-level addition, deletion, and highlighting. (3) Selective merging: Supports single question replacement / insertion, as well as batch replacement / insertion after selecting multiple questions, only writing back the questions selected by the user to the final version, avoiding full overwrite and loss of manual editing; (4) Performance optimization: The question comparison row uses a memoized component and stable callback to ensure smooth rendering of the list of hundreds of questions.

8. The method for automatic questionnaire generation and optimization based on a large language model according to claim 1, characterized in that, The specific process of high-fault-tolerant YAML parsing is as follows: First, code block extraction: use a state machine to parse Markdown code blocks character by character, and handle nested backticks and consecutive code blocks; Then, multi-level error recovery: first, preprocess and repair invalid escape sequences within double quotes; then detect indentation anomalies and complete the indentation for retry; finally, extract the line number of the error from the error message using regular expressions, delete the line, and retry. Finally, dual-format normalization: It supports both Chinese field list format and English field dictionary format, and outputs a standardized structure for front-end consumption.

9. The method for automatic questionnaire generation and optimization based on a large language model according to claim 1, characterized in that, The specific process of dialogue context management is as follows: First, the uploaded long document is divided into sliding blocks with a fixed length and overlapping window, and stored as multiple block context records. Then, LLM is called to generate summary context. Then, the YAML of the questionnaire generated in each round is persisted as a YAML context and injected with the LLM hints for the next round, realizing cross-round memorization of questionnaire content and supporting iterative dialogues that can be modified based on the current questionnaire; Finally, a greedy truncation algorithm is used to truncate historical messages based on the token budget, always retaining system messages and the latest user messages.

Images