Report generation method, device, equipment and storage medium

Abstract

This disclosure provides a report generation method, apparatus, device, and storage medium, relating to the field of artificial intelligence, particularly to technical fields such as large models and data processing. A specific implementation scheme comprises: generating a first question label description based on a first question label; obtaining and configuring a second question label corresponding to target data based on the first question label and the first question label description; performing structured processing on the target data based on the target data, the second question label, and structured format information to obtain structured data; and generating a report based on the structured data and a report template.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as large models and data processing. Background Art

[0002] In some scenarios, such as surveys, reports need to be generated based on the collected information. Surveys, short for investigation and research, involve collecting, organizing, and analyzing information to gain a deeper understanding of the research subject. Survey systems typically rely on traditional methods such as questionnaires, interviews, and manual labeling, resulting in low efficiency and accuracy. Although some survey systems use basic artificial intelligence methods such as natural language processing (NLP) for classification or analysis, these systems have limited model capabilities, making it difficult to generate accurate survey reports. Summary of the Invention

[0003] The present disclosure provides a report generation method, apparatus, device, and storage medium.

[0004] According to one aspect of the present disclosure, a report generation method is provided, comprising:

[0005] generating a first question tag description based on the first question tag;

[0006] Based on the first question tag and the first question tag description, obtaining and configuring a second question tag corresponding to the target data;

[0007] Based on the target data, the second question label, and the structured format information, performing structured processing on the target data to obtain structured data;

[0008] A report is generated based on the structured data and the report template.

[0009] According to another aspect of the present disclosure, a method for training a large language model for generating a report is provided, comprising:

[0010] Using the report generation method in any embodiment of the present disclosure, obtain output content data of the large language model at one or more target stages;

[0011] Obtaining modified content data corresponding to the output content data;

[0012] constructing a target phase sample based on the output content data and its corresponding modified content data;

[0013] A flywheel mechanism is used to fine-tune the large language model based on the target stage samples.

[0014] According to another aspect of the present disclosure, there is provided a report generating apparatus, comprising:

[0015] A description generating module, configured to generate a first question tag description based on the first question tag;

[0016] a configuration module, configured to obtain and configure a second question label corresponding to the target data based on the first question label and the first question label description;

[0017] a structuring module, configured to perform structural processing on the target data based on the target data, the second question label, and the structured format information to obtain structured data;

[0018] The report generation module is used to generate a report based on the structured data and the report template.

[0019] According to another aspect of the present disclosure, a training apparatus for a large language model for generating a report is provided, comprising:

[0020] an output acquisition module, configured to acquire output content data of the large language model at one or more target stages using the report generation device in any embodiment of the present disclosure;

[0021] A modification acquisition module, used to acquire modified content data corresponding to the output content data;

[0022] A construction module, configured to construct a target phase sample based on the output content data and its corresponding modified content data;

[0023] An adjustment module is used to fine-tune the large language model based on the target stage samples using a flywheel mechanism.

[0024] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0025] at least one processor; and

[0026] a memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0028] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0029] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements any one of the methods according to the embodiments of the present disclosure.

[0030] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0032] Figure 1 is a flowchart of a report generation method according to an embodiment of the present disclosure;

[0033] Figure 2 is a flowchart of a report generating method according to another embodiment of the present disclosure;

[0034] Figure 3 is a flowchart of a report generating method according to another embodiment of the present disclosure;

[0035] Figure 4 is a flowchart of a report generating method according to another embodiment of the present disclosure;

[0036] Figure 5 is a flowchart of a report generating method according to another embodiment of the present disclosure;

[0037] Figure 6 is a flowchart of a method for training a large language model for generating reports according to an embodiment of the present disclosure;

[0038] Figure 7 is an overall flow chart according to an embodiment of the present disclosure;

[0039] Figure 8 is a structural diagram of a report generating device according to an embodiment of the present disclosure;

[0040] Figure 9 is a structural diagram of a report generating device according to another embodiment of the present disclosure;

[0041] Figure 10 is a structural diagram of a large language model training device for generating reports according to an embodiment of the present disclosure;

[0042] Figure 11 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] Figure 1 FIG. 1 is a flow chart of a report generation method 100 according to an embodiment of the present disclosure, the method comprising:

[0045] S110, generating a first question tag description based on the first question tag;

[0046] S120: Based on the first question tag and the first question tag description, obtain and configure a second question tag corresponding to the target data;

[0047] S130: Based on the target data, the second question label, and the structured format information, perform structured processing on the target data to obtain structured data;

[0048] S140: Generate a report based on the structured data and the report template.

[0049] In the embodiments of the present disclosure, one or more first question tags can be set according to the requirements of a specific application scenario (or referred to as target requirements). A first question tag can summarize the characteristics of a certain question, and the first question tag description can describe the characteristics of the question in detail. The first question tag can be understood as a candidate question tag. Based on the first question tag and the first question tag description, a second question tag corresponding to a certain question in the target data can be selected from multiple first question tags.

[0050] Taking a survey scenario as an example: S110 may include generating first question label descriptions based on each first question label required for the survey. S120 may include acquiring and configuring second question labels corresponding to survey data based on each first question label and its corresponding first question label description. S130 may include structuring the survey data based on the survey data, its corresponding second question label, and structured format information to obtain structured data. S140 may include generating a survey report based on the structured data and a survey report template.

[0051] In an embodiment of the present disclosure, survey data needs to be collected during the survey process. For example, through online and / or offline survey questionnaires, survey forms, etc., the various questions required for the survey can be presented to the respondents, so as to collect the answers given by the respondents to each question. Due to the diversity of survey purposes, objects, business scenarios, etc., the questions required for different types of surveys may be different. Each question can have a corresponding label, i.e., a first question label. Different first question labels can summarize the characteristics of different problems, and the first question label description can describe in detail the characteristics of the problem summarized by the first question label. For example, the first question label may include "operational pain points" and "market demand". The first question label description corresponding to "operational pain points" may include "describing the problems faced by users in existing products or services, such as complex operations, insufficient performance, etc." The first question label description corresponding to "market demand" may include "defining potential needs in the target market, such as emerging technology applications or missing functions of a certain type of product."

[0052] Using artificial intelligence solutions, we can automatically set second question labels for the collected survey data based on each first question label and its corresponding first question label description. For example, the questionnaire questions in the survey data are as follows:

[0053] 1. What are the most common problems current users encounter when using similar products?

[0054] 2. What are the main differentiating features of similar products or services on the market?

[0055] Among them, question 1 corresponds to the label "operational pain points" and question 2 corresponds to the label "market demand".

[0056] The raw survey data may be unstructured or may not conform to the output format requirements for structured format information, such as structured data. Output format requirements can vary, such as a table format or a priority list. Other output format requirements are also possible and are not specifically limited in the present disclosure.

[0057] An example of a table format is as follows:

[0058] User feedback issues frequency Product functional requirements Priority Complex operation high frequency Simplified user interface high The function is not stable enough intermediate frequency Improve performance stability middle

[0059] After assigning question labels to the raw survey data, you can structure the data and its question labels to produce structured data that meets the output format requirements. For example, using a table format, you can sort the data by user feedback and functional requirements to generate a comparison table. Using a priority list, you can sort by user feedback frequency or market demand popularity to output a priority list.

[0060] Then, based on this structured data and the report template required by the researcher, a research report can be automatically generated. The report template can include the content required for the research report, such as the background of the research subject, the research purpose, the research method, the current situation analysis, the main conclusions of the research, the discussion and analysis, and the recommended countermeasures.

[0061] An example of the content of a research report is as follows:

[0062] 1. Operational pain point analysis: Summarize the main pain points of existing products and provide specific cases.

[0063] 2. Market demand overview: Provides the market's overall expectations and trend forecasts for similar product features.

[0064] 3. Recommendations and Conclusions: Based on the survey results, propose specific optimization directions, such as improving interface design, enhancing functional stability, and formulate implementation priority plans.

[0065] The disclosed embodiments can automatically set corresponding question tags for various types of target data, such as survey data, automatically generate structured data of the target data, and then flexibly generate various styles of reports, such as survey reports, based on the structured data and report templates. It can adapt to the needs of various business scenarios and improve the accuracy and completeness of the report content.

[0066] Figure 2 2 is a flow chart of a report generation method 200 according to another embodiment of the present disclosure. The method 200 can be used to implement step S110 in the report generation method 100. In one embodiment, the method 200 includes: generating a first question tag description based on a first question tag, and further includes:

[0067] S210: Generate a task based on the first question label and the received label description, and determine a first prompt word;

[0068] S220: Input the first prompt word into the large language model to obtain the output first question label description.

[0069] In the disclosed embodiment, each first question label and label description generation task can be input into the large language model through the first prompt word. The label description generation task is executed by the large language model to generate a detailed description of each first question label, i.e., the first question label description.

[0070] Taking a survey scenario as an example, S210 may include generating a task based on the first question labels required for the survey and the received label descriptions, and determining a first prompt word. S220 may include inputting the first prompt word into a large language model to output first question label descriptions corresponding to the first question labels of the survey.

[0071] In the disclosed embodiments, the large language model may include a general large language model or a large language model fine-tuned using survey training samples. The survey training samples may be prepared in advance or constructed based on the generated content of each stage of the report generation method.

[0072] In the disclosed embodiment, the first prompt word can be used to input the various question labels and label description generation tasks required for the survey into the large language model. The large language model can then perform the label description generation task and generate detailed descriptions of the various question labels.

[0073] The large language model can automatically generate detailed and accurate label descriptions for question labels, allowing the large language model to accurately understand question labels and then generate accurate content, improving the accuracy and completeness of the report content.

[0074] Furthermore, the first question label description corresponding to the first question label output by the large language model can be displayed to users, such as researchers, and the user can modify the first question label description. The large language model can also be used to generate first question label descriptions corresponding to the same first question label multiple times. By comparing the differences in the multiple generated results, the optimal label description is selected as the modified first question label description. The first question label description before modification and the first question label description after modification can constitute training samples for the survey question description stage. These training samples can be used to fine-tune the large language model to further improve the accuracy of the large language model description labels.

[0075] Figure 3 3 is a flow chart of a report generation method 300 according to another embodiment of the present disclosure. This method 300 can be used to implement step S120 in the report generation method 100. In one embodiment, the sentiment feature includes a short-term sentiment sequence feature and a long-term sentiment sequence feature. The method 300 includes: based on the first question label and its corresponding first question label description, obtaining and configuring a second question label corresponding to the target data; and further includes:

[0076] S310, determining a second prompt word according to one or more of the first question label, the first question label description, the target data, and the received label description setting task;

[0077] S320: Input the second prompt word into the large language model to obtain the second question label corresponding to the question and answer in the output target data.

[0078] In the disclosed embodiment, the target data may include individual questions and answers collected based on the individual questions. Using a second prompt, each first question label and its corresponding first question label description, along with the target data and label description setting task, may be input into the large language model. The labeling module of the large language model may select appropriate second question labels for each question and its answer in the target data from the individual first question labels.

[0079] Taking a survey scenario as an example, S310 may include determining a second prompt word based on one or more of the first question labels and their corresponding first question label descriptions, the survey data, and the received label description setting task. S320 may include inputting the second prompt word into the large language model to output second question labels corresponding to the questions and answers in the survey data.

[0080] In an embodiment of the present disclosure, the survey data may include the various questions required for the survey and the answers collected based on each question. Through a second prompt, each first question label and its corresponding first question label description, the survey data, and the label description setting task can be input into the large language model. The labeling module of the large language model can accurately understand each first question label based on the first question label description, and then select and set a second question label from each first question label for each question and answer in the survey data. A question and its answer can correspond to a question label, and different questions and their answers may correspond to the same question label. For example, the questions required for a business scenario survey include Q1, Q2, and Q3, corresponding to question labels L1, L2, and L3, respectively. The question and answer pairs in the actual collected survey data include {Q1, A1}, {Q2, A2}, and {Q3, A3}. The large language model can set label L1 for {Q1, A1}, label L2 for {Q2, A2}, and label L3 for {Q3, A3}. For example, a business scenario survey requires questions including Q4, Q5, and Q6. Q4 and Q5 are assigned the question label L4, while Q6 is assigned the question label L5. The question-answer pairs in the actual survey data collected include {Q4, A4}, {Q5, A5}, and {Q6, A6}. The large language model can assign the label L4 to {Q4, A4}, the label L5 to {Q5, A5}, and the label L6 to {Q6, A6}. Assigning labels is also called tagging.

[0081] The large language model can automatically generate accurate question labels for collected target data, such as survey data, to meet the needs of various business scenarios and improve the accuracy and completeness of report content.

[0082] Furthermore, the large language model outputs secondary question labels corresponding to the questions and answers in the target data, and allows users, such as researchers or the model, to modify the correspondence between questions and answers and question labels. The label settings before and after the modification can constitute training samples during the data collection phase. These training samples can be used to fine-tune the large language model, further improving the accuracy of the labels assigned by the large language model to the survey data.

[0083] Figure 4 4 is a flow chart of a report generation method 400 according to another embodiment of the present disclosure. This method 400 can be used to implement step S130 in the report generation method 100. In one embodiment, the method 400 includes: performing structured processing on the target data based on the target data, the second question label, and the structured format information to obtain structured data, and further includes:

[0084] S410: Determine a third prompt word based on one or more of the target data, the second question label, the second question label description corresponding to the second question label, the structured format information, and the structured task;

[0085] S420: Input the third prompt word into the large language model to perform structured processing on the target data to obtain the output structured data.

[0086] In the embodiment of the present disclosure, the target data may have specific structured format information. The structured format information may include structured format requirements, such as table format requirements, list format requirements, etc.

[0087] Taking a survey scenario as an example, S410 may include determining a third prompt word based on one or more of the survey data, a first question label corresponding to the survey data, a first question label description corresponding to the first question label, the structured format information, and a structured task. S420 may include inputting the third prompt word into the large language model, performing structured processing on the survey data, and outputting the structured data.

[0088] In the embodiments of the present disclosure, the structured format requirements of survey data for different business scenarios may be different. For example, the structured format requirements of survey data for the business scenario of the above-mentioned market research may include: table format. The survey data, the question labels corresponding to the survey data, the label descriptions corresponding to the question labels, the structured format requirements, and the structured tasks can be input into the large language model through the third prompt word. The large language model can accurately understand the question labels based on the label descriptions, and perform structured tasks, and perform structured processing on the survey data that have been set with question labels according to the structured format requirements to obtain structured data. For example, the survey data {Q1, A1} with label L1, the survey data {Q2, A2} with label L2, and the survey data {Q3, A3} with label L3 are structured.

[0089] The large language model can be used to convert target data, such as unstructured data in survey data, into structured data that meets business needs, providing support for the subsequent generation of accurate reports that meet business needs, such as survey reports, and improving the accuracy and completeness of the report content.

[0090] Furthermore, the structured data output by the large language model can be displayed to users, such as researchers, and users can modify the structured data, for example, by changing the content of certain items in a survey form output by the model. The pre- and post-modification structured data can serve as training samples for the structuring phase. These training samples can be used to fine-tune the large language model, further improving its accuracy in structuring data.

[0091] Figure 5 is a flow chart of a report generation method 500 according to another embodiment of the present disclosure. The method 500 may be used to implement step S140 in the report generation method 100. In one embodiment, the method 500 includes: generating a report based on the structured data and the report template, further including:

[0092] S510, determining a fourth prompt word according to one or more of the structured data, the report template, and the report generation task;

[0093] S520: Generate the report based on the structured data and the report template by inputting the fourth prompt word into the large language model.

[0094] In the disclosed embodiment, the fourth prompt word can be used to input structured data, a report template, and a report generation task into the large language model. The large language model can then execute the report generation task, selecting the data required for the report template from the structured data, filling the selected data into the report template, and generating the report.

[0095] Taking the survey scenario as an example: S510 may include determining a fourth prompt word based on the structured data, the survey report template, and one or more of the survey report generation tasks; S520 may include inputting the fourth prompt word into the large language model and generating the survey report based on the structured data and the survey report template.

[0096] In the disclosed embodiment, the report templates for the targets corresponding to different business needs may be different. For example, research report template 1 includes the background of the research object, the purpose of the research, the research method and the main conclusions. Research report template 2 includes the background of the research object, the current situation analysis, the main conclusions of this research, the discussion and analysis, and the suggested countermeasures. The structured data, the research report template and the research report generation task can be input into the big language model through the fourth prompt word. The big language model can execute the research report generation task, select the data required for the research report template from the structured data, fill the selected data into the research report template, and generate the research report.

[0097] The large language model can be used to fill structured data that meets business needs into report templates, flexibly generating various styles of reports such as research reports.

[0098] Furthermore, reports generated by the large language model, such as research reports, can be displayed to users, such as researchers, allowing them to modify the report's content. The pre- and post-modification reports can serve as training samples for the report generation phase. These training samples can be used to fine-tune the large language model, enabling it to flexibly generate research reports.

[0099] In one embodiment, the large language model is fine-tuned using a flywheel mechanism based on target stage samples; the target stage samples include output content data of the large language model at one or more target stages and modified content data corresponding to the output content data. The flywheel mechanism can include a dynamic process of continuous optimization and improvement of model performance during model training.

[0100] Taking the survey scenario as an example, the large language model uses a flywheel mechanism to fine-tune based on survey stage samples; wherein, the survey stage samples include the output content data of the large language model in one or more stages of the survey and the modified content data corresponding to the output content data.

[0101] In the disclosed embodiments, training samples (referred to as survey stage samples) can be obtained in any one or more survey stages, such as the survey question description stage, the data collection stage, the structuring stage, and the report generation stage. Using the flywheel mechanism, based on the training samples from any of the aforementioned target stages, such as the survey stage, the network parameters of the large language model can be fine-tuned to obtain a model (referred to as the survey model) that is more suitable for various business needs, such as survey business needs. This helps improve the accuracy and completeness of the reports generated by the model.

[0102] Figure 6 FIG. 6 is a flow chart of a method 600 for training a large language model for generating a report according to an embodiment of the present disclosure. The method includes:

[0103] S610: Using the report generation method in any of the above embodiments, obtain output content data of the large language model at one or more target stages;

[0104] S620: Obtain modified content data corresponding to the output content data;

[0105] S630: Construct a target phase sample based on the output content data and its corresponding modified content data;

[0106] S640: Use a flywheel mechanism to fine-tune the large language model based on the target stage samples.

[0107] In the disclosed embodiments, the target phase may include a problem description phase, a data collection phase, a structuring phase, and a report generation phase. In one or more phases, the output content data of the large language model may support modification. After manually or automatically modifying the output content data, the modified content data corresponding to the output content data may be obtained. Using a certain target phase, the output content data of the large language model and its corresponding modified content data may be used to construct a training sample for that target phase, i.e., a target phase sample.

[0108] Taking a survey scenario as an example: S610 may include using the aforementioned report generation method to obtain output content data of the large language model at one or more survey stages. S620 may include obtaining modified content data corresponding to the output content data. S630 may include constructing survey stage samples based on the output content data and its corresponding modified content data. S640 may include fine-tuning the large language model based on the survey stage samples using a flywheel mechanism.

[0109] In an embodiment of the present disclosure, the survey stage may include any one or more of a survey question description stage, a survey data collection stage, a structuring stage, and a survey report generation stage. In one or more stages, the output content data of the large language model may support modification. After manual or automatic modification by the model, the modified content data corresponding to the output content data may be obtained. Using a certain survey stage, the output content data of the large language model and its corresponding modified content data, the training sample of the survey stage, i.e., the survey stage sample, may be constructed. Fine-tuning the large language model using various target stage samples, such as survey stage samples, may improve the accuracy of the output content of the large language model in various target stages, such as the survey stage, and thereby improve the accuracy of reports, such as survey reports.

[0110] In one embodiment, the output content data of the one or more stages includes one or more of the following:

[0111] In the problem description phase, the first problem label description is generated;

[0112] During the data collection phase, the second question label is set for the target data;

[0113] In the structuring stage, the target data is subjected to structural processing to obtain structured data;

[0114] During the report generation phase, a report is generated.

[0115] Take the survey scenario as an example: the output content data of one or more survey stages: the question label description generated in the survey question description stage; the question labels set for the survey data in the survey data collection stage; the structured data obtained by structured processing of the survey data in the structuring stage; and the survey report generated in the survey report generation stage.

[0116] In some examples, during the research question description phase, the output content data of the large language model may include label descriptions generated for the question labels. The modified content data corresponding to the output content data may include the modified label descriptions. The first label description before modification and the first label description after modification may constitute training samples for the research question description phase. These training samples can be used to fine-tune the large language model, further improving the accuracy of the large language model's label descriptions.

[0117] During the survey data collection phase, the output content data of the large language model may include question labels corresponding to questions and answers in the survey data. Modified content data corresponding to the output content data may include label settings after the researcher or model has modified the correspondence between questions and answers and question labels. The pre- and post-modification label settings can constitute training samples for the data collection phase. These training samples can be used to fine-tune the large language model, further improving the accuracy of the labels assigned by the large language model to the survey data.

[0118] During the structuring phase, the output content data of the large language model may include structured data. The modified content data corresponding to the output content data may include the structured data modified by the researcher or the model. The pre-modified and post-modified structured data constitute training samples for the structuring phase. These training samples can be used to fine-tune the large language model, further improving its accuracy in structuring data.

[0119] During the research report generation phase, the output content data of the large language model may include the research report. The modified content data corresponding to the output content data may include the research report modified by the researcher or the model. The pre- and post-modification research reports can constitute training samples for the report generation phase. These training samples can be used to fine-tune the large language model, enabling it to flexibly generate research reports.

[0120] By fine-tuning the large language model using training samples from different stages, the accuracy and flexibility of the model's output can be improved, thereby obtaining reports that meet business needs, such as research reports.

[0121] The relevant survey systems have the following main shortcomings:

[0122] 1. Lack of intelligent labeling and feedback: Data labels in related systems often need to be generated manually or semi-automatically, which makes it impossible to achieve efficient and dynamic label adjustment, and dynamic integration of user feedback is relatively difficult.

[0123] 2. Inability to generate multi-stage structured information: During complex research processes, information needs to be adjusted and structured multiple times, but the relevant systems lack the corresponding multi-stage processing and data flow mechanisms.

[0124] 3. Low response efficiency: When faced with real-time data analysis and user feedback, relevant systems often respond slowly, making it difficult to provide accurate research reports and analysis results in a short period of time.

[0125] 4. Poor data accuracy and consistency: Due to the lack of intelligent research methods based on advanced LLM models, the accuracy and consistency of data processing in relevant systems need to be improved.

[0126] Relevant survey systems often have deficiencies when faced with complex data requirements, dynamic question adjustments, and multi-dimensional data label generation, resulting in limited management, analysis, and decision-making support capabilities for survey data. The disclosed embodiments can solve problems such as low data processing efficiency, difficulty in collecting structured information, and difficulty in effectively integrating user feedback in current survey systems. The survey report generation method of the disclosed embodiment can be used in a survey system based on a large language model (LLM), and efficient and accurate data label generation and survey result output are achieved by integrating user interaction and LLM's multi-stage processing flow. The system is suitable for a variety of scenarios such as business surveys, market research, and customer feedback analysis, and helps to achieve efficient data collection and analysis in complex data environments.

[0127] The disclosed embodiment of the survey system, based on the Large Language Model (LLM), leverages the LLM's multi-level data processing and feedback generation capabilities to provide complete survey process support, enabling efficient data flow and label generation, ultimately producing accurate survey reports. The disclosed embodiment achieves automated management of the entire process of data collection, structuring, analysis, and output through the following technical solutions. Figure 7 This is a diagram of a survey system based on the LLM. The block diagram, from input to output, illustrates the workflow of each stage of the survey system, including the workflow nodes of multiple stages such as "problem definition," "data collection," "information structuring," and "overall summary." Figure 7 It also includes a feedback optimization mechanism, where the system continuously optimizes the accuracy of data labels and structured outputs through cyclic feedback at each stage. Figure 7 The role of the LLM model in each stage can also be demonstrated. For example, the specific roles of LLM in problem definition, data collection, structuring and summary stages are "LLM definition label", "LLM labeling", "LLLM structured information output" and "LLM overall report output".

[0128] Detailed descriptions of each stage of the survey method performed by the survey system are as follows:

[0129] S701. Problem definition phase: After the survey data is input through the "Problem definition phase" node, the survey system first enters the problem definition phase. LLM generates a preliminary label description (see output 1, Output1) based on the input data content, such as the business definition label (see user action 1, Useraction1). LLM can also modify the output (Output1). In addition, through the flywheel mechanism such as data flywheel 1, the function of LLM definition label can be iteratively optimized. The main function of this stage is to preliminarily define the business scope and data labels of the survey. The flywheel mechanism in the LLM field can include a data flywheel. The data flywheel is a cyclic mechanism that optimizes the performance of the LLM by continuously collecting, utilizing and feeding back data.

[0130] S702. Data collection and analysis phase: Users submit further questions and / or answers and other survey data through the "Survey Data Collection" node in this phase (see User action 2). LLM will classify them into preliminary questions, review questions, or new requirements, and generate initial analysis results (see Output 2). The survey data is passed to the LLM labeling module, and the system generates label information based on business needs. In addition, through a flywheel mechanism such as Data Flywheel 2, the LLM labeling function can be iteratively optimized. The main function of this phase is for LLM to generate data labels for the classified survey data based on business needs.

[0131] S703. Information Structuring Phase: During this phase, the system organizes and structures the collected data. LLM combines business requirements and contextual information (see User Action 3) to generate structured data that conforms to the output format (see Output 3), optimizing subsequent analysis and presentation. Furthermore, through flywheel mechanisms such as Data Flywheel 3, LLM's structuring capabilities can be iteratively optimized. The role of LLM in this phase is to transform unstructured data into structured output that meets business requirements.

[0132] S704. Research Summary and Output Phase: LLM performs a final aggregation and summary of all data (see User Action 4), generating a research report or analysis results (see Output 4). Furthermore, through flywheel mechanisms such as Data Flywheel 4, LLM's structured functionality can be iteratively optimized. LLM's structured information output module further processes and refines the content. LLM's role in this phase ensures the completeness and accuracy of the report.

[0133] S705. Feedback Optimization Mechanism: The system incorporates a flywheel mechanism, providing feedback and optimizing the accuracy of data labels and structured information at each stage. Through multiple rounds of iterative training, the model's adaptability to various business problems is continuously improved.

[0134] The system of this disclosed embodiment incorporates LLM's automated feedback and optimization mechanisms, ensuring continuous optimization and improvement of data labels and output results at every step of data processing. Furthermore, this disclosed embodiment utilizes a custom flywheel mechanism to reduce manual intervention and ensure the model's self-learning capabilities, thereby saving storage space and accelerating processing speed.

[0135] The survey system of the embodiment of the present disclosure is widely applicable to various data-intensive survey scenarios. For example, the application scenarios may include business surveys, market trend analysis, customer feedback management and other fields.

[0136] Figure 8 FIG. 8 is a schematic structural diagram of a report generating apparatus 800 according to an embodiment of the present disclosure. The apparatus 800 may include:

[0137] A description generating module 810 is configured to generate a first question tag description based on the first question tag;

[0138] A configuration module 820 is configured to obtain and configure a second question tag corresponding to the target data based on the first question tag and the first question tag description;

[0139] a structuring module 830 for performing structural processing on the target data based on the target data, the second question label, and the structured format information to obtain structured data;

[0140] The report generation module 840 is configured to generate a report based on the structured data and a report template.

[0141] Figure 9 : This is a schematic diagram of the structure of a report generation device 900 according to another embodiment of the present disclosure. The device 900 includes: a description generation module 910, a configuration module 920, a structuring module 930, and a report generation module 940. The functions of the above modules can refer to the functions of the modules of the report generation device in the above embodiment. In one embodiment, the description generation module 910 may include:

[0142] A first prompt word submodule 911 is configured to generate a task based on the first question label and the received label description, and determine a first prompt word;

[0143] The description generation submodule 912 is used to input the first prompt word into the large language model to obtain the output first question label description.

[0144] In one embodiment, the configuration module 920 includes:

[0145] A second prompt word submodule 921 is configured to determine a second prompt word according to one or more of the first question label, the first question label description, the target data, and the received label description setting task;

[0146] A submodule 922 is provided for inputting the second prompt word into the large language model to obtain the second question label corresponding to the question and answer in the output target data.

[0147] In one embodiment, the structuring module 930 includes:

[0148] A third prompt word submodule 931 is configured to determine a third prompt word based on one or more of the target data, the second question label, a second question label description corresponding to the second question label, the structured format information, and the structured task;

[0149] The structuring submodule 932 is configured to input the third prompt word into the large language model to perform structuring processing on the target data to obtain the output structured data.

[0150] In one embodiment, the report generation module 940 includes:

[0151] A fourth prompt word submodule 941 is configured to determine a fourth prompt word according to one or more of the structured data, the report template, and the report generation task;

[0152] The report generation submodule 942 is configured to generate the report based on the structured data and the report template by inputting the fourth prompt word into the large language model.

[0153] In one embodiment, the large language model uses a flywheel mechanism to perform fine-tuning based on target stage samples; wherein the target stage samples include output content data of the large language model at one or more target stages and modified content data corresponding to the output content data.

[0154] Figure 10 FIG. 1 is a schematic structural diagram of a large language model training apparatus 1000 for generating reports according to an embodiment of the present disclosure. The apparatus 1000 may include:

[0155] An output acquisition module 1010 is configured to acquire output content data of the large language model at one or more target stages using the report generation apparatus in the above embodiment;

[0156] A modification acquisition module 1020 is used to acquire modified content data corresponding to the output content data;

[0157] A construction module 1030 is configured to construct a target phase sample based on the output content data and its corresponding modified content data;

[0158] The adjustment module 1040 is configured to fine-tune the large language model based on the target stage sample using a flywheel mechanism.

[0159] In one embodiment, the output content data of the one or more stages includes one or more of the following:

[0160] In the problem description phase, the first problem label description is generated;

[0161] During the data collection phase, the second question label is set for the target data;

[0162] In the structuring stage, the target data is subjected to structural processing to obtain structured data;

[0163] During the report generation phase, a report is generated.

[0164] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0165] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0166] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0167] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0168] like Figure 11As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 11011 into a random access memory (RAM) 1103. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0169] Various components in device 1100 are connected to I / O interface 1105, including an input unit 1106, such as a keyboard and mouse; an output unit 1107, such as various types of displays and speakers; a storage unit 1108, such as a magnetic disk and optical disk; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0170] The computing unit 1101 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the report generation method and / or the training method of the large language model for generating the report. For example, in some embodiments, the report generation method and / or the training method of the large language model for generating the report can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the report generation method and / or the training method of the large language model for generating the report described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured in any other appropriate manner (eg, by means of firmware) to execute the report generation method and / or the training method of the large language model for generating the report.

[0171] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0172] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0173] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0175] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0176] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0177] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0178] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A report generation method, comprising: generating a first question tag description based on the first question tag; Based on the first question tag and the first question tag description, obtaining and configuring a second question tag corresponding to target data; wherein the target data includes each question and answers collected based on each question; the second question tag is selected from each of the first question tags for each question and its answer in the target data; Based on the target data, the second question label, and the structured format information, performing structured processing on the target data to obtain structured data; A report is generated based on the structured data and the report template.

2. The method according to claim 1, wherein Generating a first question tag description based on the first question tag includes: Generate a task based on the first question label and the received label description, and determine a first prompt word; The first prompt word is input into a large language model to obtain the output first question label description.

3. The method according to claim 1, wherein The acquiring and configuring a second question tag corresponding to the target data based on the first question tag and the first question tag description includes: Determining a second prompt word according to one or more of the first question label, the first question label description, the target data, and the received label description setting task; By inputting the second prompt word into the large language model, the second question label corresponding to the question and answer in the output target data is obtained.

4. The method according to any one of claims 1 to 3, wherein The step of performing structural processing on the target data based on the target data, the second question label, and the structured format information to obtain structured data includes: determining a third prompt word according to one or more of the target data, the second question label, a second question label description corresponding to the second question label, the structured format information, and a structured task; The target data is structured by inputting the third prompt word into a large language model to obtain the output structured data.

5. The method according to any one of claims 1 to 3, wherein Generating a report based on the structured data and the report template includes: determining a fourth prompt word according to one or more of the structured data, the report template, and the report generation task; The report is generated based on the structured data and the report template by inputting the fourth prompt word into a large language model.

6. The method according to claim 2 or 3, wherein the large language model is fine-tuned based on target stage samples using a flywheel mechanism; The target stage sample includes output content data at one or more target stages based on the large language model and modified content data corresponding to the output content data.

7. A method for training a large language model for generating reports, comprising: Using the method according to any one of claims 1 to 6, obtaining output content data of the large language model at one or more target stages; Acquire modified content data corresponding to the output content data; constructing a target phase sample based on the output content data and its corresponding modified content data; A flywheel mechanism is adopted to fine-tune the large language model based on the target stage samples.

8. The method according to claim 7, wherein: The output content data of the one or more stages includes one or more of the following: In the problem description phase, the first problem label description is generated; During the data collection phase, the second question label is set for the target data; In the structuring stage, the target data is subjected to structuring processing to obtain structured data; During the report generation phase, a report is generated.

9. A report generating device comprising: A description generating module, configured to generate a first question tag description based on the first question tag; a configuration module, configured to obtain and configure a second question tag corresponding to target data based on the first question tag and the first question tag description; wherein the target data includes each question and answers collected based on each question; and the second question tag is selected from each of the first question tags for each question and its answer in the target data; a structuring module, configured to perform structural processing on the target data based on the target data, the second question label, and the structured format information to obtain structured data; A report generation module is used to generate a report based on the structured data and a report template.

10. The device according to claim 9, wherein The description generation module includes: A first prompt word submodule, configured to generate a task according to the first question label and the received label description, and determine a first prompt word; The description generation submodule is used to input the first prompt word into the large language model to obtain the output first question label description.

11. The device according to claim 9, wherein The configuration module includes: a second prompt word submodule, configured to determine a second prompt word according to one or more of the first question label, the first question label description, the target data, and a received label description setting task; A submodule is set up to obtain the second question label corresponding to the question and answer in the output target data by inputting the second prompt word into the large language model.

12. The device according to any one of claims 9 to 11, wherein The structured module includes: a third prompt word submodule, configured to determine a third prompt word based on one or more of the target data, the second question label, a second question label description corresponding to the second question label, the structured format information, and a structured task; The structuring submodule is configured to input the third prompt word into a large language model to perform structuring processing on the target data to obtain the output structured data.

13. The device according to any one of claims 9 to 11, wherein The report generation module includes: a fourth prompt word submodule, configured to determine a fourth prompt word according to one or more of the structured data, the report template, and the report generation task; The report generation submodule is configured to generate the report based on the structured data and the report template by inputting the fourth prompt word into the large language model.

14. The apparatus according to claim 10 or 11, wherein the large language model is fine-tuned based on target stage samples using a flywheel mechanism; The target stage sample includes output content data at one or more target stages based on the large language model and modified content data corresponding to the output content data.

15. A training apparatus for a large language model for generating reports, comprising: an output acquisition module, configured to acquire output content data of the large language model at one or more target stages using the apparatus according to any one of claims 9 to 14; A modification acquisition module, configured to acquire modified content data corresponding to the output content data; A construction module, configured to construct a target phase sample based on the output content data and its corresponding modified content data; An adjustment module is used to fine-tune the large language model based on the target stage samples using a flywheel mechanism.

16. The device according to claim 15, wherein The output content data of the one or more stages includes one or more of the following: In the problem description phase, the first problem label description is generated; During the data collection phase, the second question label is set for the target data; In the structuring stage, the target data is subjected to structuring processing to obtain structured data; During the report generation phase, a report is generated.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images