Method and apparatus for generating survey question, and storage medium

Abstract

Provided are a method and apparatus for generating a survey question, and a storage medium. The method comprises: issuing a first question, the first question being one of multiple preset questions; acquiring a first answer corresponding to the first question; and on the basis of the first answer, determining a second question from among the multiple preset questions, the second question being related to the first question.

Description

Methods, devices, and storage media for generating research questions

[0001] This disclosure claims priority to Chinese patent application No. 202411885353.4, filed on December 19, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of artificial intelligence technology, and in particular to a method, apparatus and storage medium for generating research questions. Background Technology

[0003] Currently, research activities mainly involve researchers asking interviewees to fill out paper questionnaires in public places or conducting interviews with interviewees via telephone, the internet, or other means to complete the questionnaires. Summary of the Invention

[0004] Firstly, this disclosure provides a method for generating survey questions. The method includes: issuing a first question, which is one of a plurality of preset questions; obtaining the answer to the first question; and determining a second question from the plurality of preset questions based on the answer to the first question, wherein the second question is related to the first question.

[0005] In one design, multiple pre-defined questions are divided into multiple levels. The first question is located at the first level of the multiple levels, and the second question is located at the second level of the multiple levels. The second question is related to the first question in the following ways: the first level and the second level are at the same level, or the second level is the next level after the first level.

[0006] In one design, a pre-defined question corresponds to at least one category label. Determining a second question from multiple pre-defined questions based on the answer to the first question includes: determining a target label based on the answer to the first question and a first level; and determining the second question based on the target label.

[0007] In one design, determining the target label based on the answer to the first question and the first level includes: parsing the answer to the first question using a large language model to determine at least one candidate label, wherein the large language model is used to generate a new question based on the question answer; and determining the target label from the at least one candidate label based on the first level.

[0008] In one design, the method further includes: obtaining a problem survey pattern. Based on a first level, determining a target label from at least one candidate label includes: if the problem survey pattern is a deep survey pattern, then determining the target label based on the category labels in the next level of the first level and at least one candidate label; if the problem survey pattern is a broad survey pattern, then determining the target label based on the category labels in the first level and at least one candidate label.

[0009] In one design, if the problem survey mode is a breadth survey mode, the target label is determined based on the category label in the first level and at least one candidate label, including: if there is a label in at least one candidate label that is the same as the category label in the first level, then the same label is taken as the target label; if there is no label in at least one candidate label that is the same as the category label in the first level, then the target label is determined based on the category label in the next level of the first level and at least one candidate label.

[0010] In one design, determining a second question from multiple preset questions based on the answer to a first question includes: if the question research mode is an in-depth research mode, and the first level is not the last level among multiple levels, then determining the second question from multiple preset questions based on the answer to the first question.

[0011] Secondly, embodiments of this disclosure provide a survey question generation apparatus, which includes an acquisition module and a processing module;

[0012] The processing module is used to issue a first question, which is one of a plurality of preset questions; the acquisition module is used to acquire the answer to the first question; the processing module is also used to determine a second question from the plurality of preset questions based on the answer to the first question, wherein the second question is related to the first question.

[0013] In one design, multiple pre-defined questions are divided into multiple levels. The first question is located in the first level of the multiple levels, and the second question is located in the second level of the multiple levels. The second question is related to the first question in the following ways: the first level and the second level are the same level, or the second level is the next level after the first level.

[0014] In one design, a preset question corresponds to at least one category label. The processing module is used to determine a target label based on the answer to a first question and a first level; the processing module is also used to determine a second question based on the target label.

[0015] In one design, the processing module is used to parse the answer to the first question using a large language model to determine at least one candidate label, and the large language model is used to generate a new question based on the question answer; the processing module is also used to determine a target label from at least one candidate label based on the first level.

[0016] In one design, the acquisition module is used to acquire the problem survey mode; the processing module is used to determine the target label based on the category label in the next level of the first level and at least one candidate label if the problem survey mode is a deep survey mode; the processing module is also used to determine the target label based on the category label in the first level and at least one candidate label if the problem survey mode is a broad survey mode.

[0017] In one design, the processing module is configured to, if at least one candidate label contains a label that is the same as the category label in the first level, then use the same label as the target label; the processing module is further configured to, if at least one candidate label does not contain a label that is the same as the category label in the first level, then determine the target label based on the category label in the next level of the first level and at least one candidate label.

[0018] In one design, the processing module is configured to determine a second question from multiple preset questions based on the answer to the first question if the question survey mode is an in-depth survey mode and the first level is not the last level among multiple levels.

[0019] Thirdly, embodiments of this disclosure provide a survey question generation apparatus, the apparatus comprising: a processor and a memory; the processor and the memory being coupled; the memory being used to store one or more programs, the one or more programs including computer-executable instructions, wherein when the survey question generation apparatus is running, the processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect and any of the designs in the first aspect.

[0020] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect and any implementation thereof.

[0021] Fifthly, embodiments of this disclosure provide a chip including a processor and a communication interface coupled to the processor, the processor being configured to run computer programs or instructions to implement the methods described in the first aspect and any of the designs in the first aspect.

[0022] In a sixth aspect, embodiments of this disclosure provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods described in the first aspect and any of the designs in the first aspect. Attached Figure Description

[0023] Figure 1 is a flowchart illustrating a method for generating survey questions according to some embodiments.

[0024] Figure 2 is a flowchart illustrating another method for generating research questions according to some embodiments.

[0025] Figure 3 is a schematic diagram of an example of a multi-level system according to some embodiments.

[0026] Figure 4 is a schematic diagram of another example of multiple levels according to some embodiments.

[0027] Figure 5 is a flowchart illustrating another method for generating survey questions according to some embodiments.

[0028] Figure 6 is a flowchart illustrating another method for generating survey questions according to some embodiments.

[0029] Figure 7 is a flowchart illustrating another method for generating survey questions according to some embodiments.

[0030] Figure 8 is a schematic diagram of a survey question generation device according to some embodiments.

[0031] Figure 9 is a schematic diagram of the structure of another survey problem generation device according to some embodiments.

[0032] Figure 10 is a conceptual partial view of a computer program product according to some embodiments. Detailed Implementation

[0033] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0034] The terms “first” and “second” in this disclosure and in the claims are used to distinguish different objects, not to describe a particular order of objects.

[0035] Furthermore, the terms “comprising” and “having”, and any variations thereof, used in the description of this disclosure are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or apparatus.

[0036] Furthermore, in the embodiments disclosed herein, expressions such as "exemplarily" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design described as "exemplarily" or "for example" in this disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of expressions such as "exemplarily" or "for example" is intended to present concepts in a detailed manner.

[0037] Currently, research activities mainly involve researchers asking respondents to fill out paper questionnaires in public places or conducting interviews via telephone, internet, or other means to complete the questionnaires. However, this method may be prone to bias during the research process, and the research results are often not accurate enough.

[0038] With the development of modern technology, especially the emergence of 5G, 5G-Advanced, and other high-speed network technologies, the connectivity between devices has been enhanced, enabling the network connection of a large number of Internet of Things (IoT) devices. This lays a solid foundation for the digital and intelligent transformation of industries.

[0039] Meanwhile, how to effectively utilize big data and intelligent technologies to generate survey questions, gain a deeper understanding of the true thoughts and needs of the survey respondents, ensure that the collected information is representative and reliable, and thus provide a strong basis for decision-making, and how to make the survey structure more accurate have also become urgent problems to be solved.

[0040] Currently, when generating survey questions, index-based building and retrieval systems are typically used to search knowledge bases and generate matching survey questions.

[0041] Indexes can be divided into forward indexes and inverted indexes. Forward indexes have a relatively simple structure, but require scanning all documents during retrieval, leading to inefficiency. The core of an inverted index is mapping keywords in a document to a list of documents containing that keyword, enabling fast retrieval of document sets containing specific keywords. Afterward, the document set can be converted into vector form, and the text can be processed using a vector space model to calculate the similarity between documents, thereby evaluating the relevance of the query results and improving query accuracy and efficiency.

[0042] However, this method often neglects the understanding of users' implicit intentions and the handling of contextual relationships, making it difficult to distinguish the different meanings of the same words in different contexts. The research questions generated are weakly related to user responses, failing to provide logically coherent and accurate information. Research questions generated using inverted indexes cannot fully understand respondents' answers, and have limitations in handling synonyms, near-synonyms, and contextual changes in respondents' responses. This often results in inaccurate matching between the generated research questions and respondents' answers, leading to discrepancies between the collected information and reality. Processing text using a vector space model requires setting a similarity threshold. The similarity threshold has a significant impact on the generation of research questions. A threshold that is too small leads to a loss of effective information; a threshold that is too large allows a large influx of invalid information, making it difficult to strike a balance.

[0043] In some embodiments, a knowledge graph is constructed through knowledge extraction and structuring processing, and pre-defined research questions are generated from the knowledge graph. Knowledge extraction extracts "entity-relationship-entity" or "entity-attribute-attribute value" information from unstructured data. Structuring processing uses natural language processing and semantic analysis to transform these entities and relationships into a structured data format and stores it in the knowledge graph, thereby improving the accuracy and relevance of the generated research questions. Subsequently, keyword matching and graph structure traversal strategies can be combined to retrieve information from the knowledge graph, further improving the efficiency and accuracy of generating research questions.

[0044] However, this approach requires complex data cleaning, structuring, and semantic parsing, which not only consumes significant manpower and time but also easily introduces irrelevant noise data, interfering with users' retrieval of relevant results. Furthermore, existing knowledge graphs are primarily built around entities and relationships, but they often fail to adequately model or cover research questions across different industries, resulting in low query accuracy and efficiency. Moreover, queries within complex graph structures, such as those with broad scope or complex paths, are extremely inefficient, often resulting in timeouts and no results being returned.

[0045] In some embodiments, certain technologies attempt to combine large language models for knowledge base construction. However, these models often fail to delve deeply into research, lack specialized information, and cannot propose targeted research questions and solutions. Furthermore, they suffer from inherent problems of large language models, such as producing inaccurate or unrealistic outputs, leading to incorrect research results.

[0046] Therefore, regardless of whether a knowledge base is built based on index-based construction and retrieval systems, constructed through knowledge extraction and structured processing to create a knowledge graph, or combined with a large language model for knowledge base construction, the generated survey questions all have certain limitations. There is a semantic discrepancy between the respondents' answers and the system's understanding. Consequently, when conducting surveys, the generated survey questions have a weak correlation with the respondents' answers, resulting in lower survey quality.

[0047] Therefore, how to make the generated survey questions more relevant to user responses has become a pressing technical problem that needs to be solved.

[0048] To address the aforementioned technical problems, this disclosure provides a method for generating survey questions. In this method, a first question can be posed, which is one of multiple preset questions. Then, the answer to the first question is obtained. Based on the answer to the first question, a second question can be determined from the multiple preset questions, and the second question is related to the first question. This strengthens the correlation between the second question and the answer to the first question, allows for understanding the implicit intent of the answer, and makes the survey questions more precise, thereby improving the efficiency and quality of the dialogue and making the survey results more accurate.

[0049] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0050] In this embodiment of the disclosure, a preset knowledge base can be constructed. Then, questions are posed based on the relationship between questions and answers in the preset knowledge base.

[0051] The construction of the knowledge base will be introduced below. As shown in Figure 1, Figure 1 is a flowchart illustrating a method for generating survey questions according to some embodiments. Knowledge data, multiple preset questions, and preset path information are acquired. The preset path information is used to indicate the data structure of the preset knowledge base. Based on the knowledge data, multiple preset questions, and preset structural relationships, the preset knowledge base is constructed.

[0052] In one design, knowledge data includes, but is not limited to, industry standard knowledge data, guideline knowledge data, policy and regulatory knowledge data, and academic paper knowledge data.

[0053] It should be noted that this disclosure does not limit the method of acquiring knowledge data. For example, databases and official websites can be accessed. Another example is that the device for generating survey questions may pre-store knowledge data.

[0054] In one design, the device for generating research questions pre-stores multiple preset questions and preset structural relationships.

[0055] In one implementation, entity extraction is performed on the knowledge data to obtain multiple knowledge units. Then, category labels are obtained based on these knowledge units. A pre-defined knowledge base is constructed based on the category labels, the multiple knowledge units, multiple preset questions, and preset path information.

[0056] In one design, knowledge data is parsed to extract initial text information, which includes, but is not limited to, text content, images, and tables. This initial text information is then preprocessed and segmented into knowledge units to obtain multiple knowledge units.

[0057] It should be noted that the knowledge data includes documents in various formats, and this disclosure does not limit the method of document parsing. For example, the Apache PDFBox or PyPDF2 library can be used to parse portable document format (PDF) documents. Another example is the Apache POI or python-docx library for parsing DOCX documents. In this way, initial text information can be extracted from documents of various formats. For image content within the initial text information, optical character recognition (OCR) technology can be used to extract text from the images, supporting the processing of complex mixed text and image data.

[0058] It should be noted that this disclosure does not limit the methods of data preprocessing. For example, regular expressions can be used to remove special characters from the text. Another example is using a natural language processing library to filter out irrelevant information and apply rules to standardize the text. The natural language processing library can be a Natural Language Toolkit (NLTK) library.

[0059] It should be noted that this disclosure does not limit the method of knowledge segmentation. For example, rule-based strategies can be used for natural segmentation, such as segmenting by paragraphs, sentences, or structured headings and subheadings. Another example is that for text with relatively standard formatting, segmentation can be performed by setting a fixed number of characters or words. Yet another example is using natural language processing techniques, such as word vectors or sentence vectors, to calculate the semantic similarity of text blocks and then aggregating the text blocks based on this semantic similarity.

[0060] It should be noted that knowledge units support high customization and dynamic updates. This allows the pre-built knowledge base to continuously evolve with industry developments and the emergence of new knowledge, and information units can be added or modified according to detailed needs to reflect the latest market trends and technological advancements.

[0061] In one design, knowledge units are categorized to obtain category labels for each knowledge unit.

[0062] For example, knowledge units can be classified based on topic models to obtain category labels for each knowledge unit among multiple knowledge units.

[0063] For example, the topic model is a latent dirichlet allocation (LDA) model.

[0064] It should be noted that topic modeling is a statistical model used to automatically discover and extract potential topics from a collection of documents.

[0065] Understandably, category labels are used to match the first question with the corresponding answer to the first question.

[0066] In one design, the preset path information includes multiple preset sub-paths. One of the preset sub-paths corresponds to two initial preset nodes, and each initial preset node corresponds to a node identifier. The node identifier is used to indicate the position of the initial preset node.

[0067] The initial preset nodes include a preset root node, preset leaf nodes, and preset intermediate nodes. The preset root node is the starting node of the preset knowledge base, the preset leaf nodes are the ending nodes of the preset knowledge base, and the preset intermediate nodes are the direct successors of any initial preset node in the preset knowledge base except for the preset leaf nodes. The preset sub-paths include a preset first path and a preset second path. The preset first path corresponds to one preset root node and one preset intermediate node, and the preset second path corresponds to one preset intermediate node and one preset leaf node.

[0068] Understandably, an initial preset node corresponds to a node identifier, which allows for quick identification and referencing of the initial preset node within the preset knowledge base.

[0069] In one design, category labels, multiple knowledge units, and multiple preset questions are assigned to initial preset nodes, resulting in multiple preset nodes. Based on these multiple preset nodes, a preset knowledge base is obtained.

[0070] For example, a preset question corresponds to a priority level, which indicates the level of detail required for the preset question. Based on the priority of the preset question, it is assigned to an initial preset node. Then, category labels and knowledge units are assigned to the initial preset nodes corresponding to the preset questions, resulting in multiple preset nodes.

[0071] Understandably, the pre-defined knowledge base organizes knowledge in a hierarchical structure, enabling multi-level classification and progressive refinement. This allows users to quickly and accurately find information by naturally and intuitively delving deeper into the layers of the problem.

[0072] It should be noted that the knowledge unit at each level in the multiple levels is a solution to the problem at the level above.

[0073] In some embodiments, a preset data structure can be obtained to store and manage a preset knowledge base.

[0074] It should be noted that this disclosure does not limit the method of storing and managing the pre-defined knowledge base. For example, it may be based on a relational database. Or, it may be based on a NoSQL database. Or, it may be based on a graph database. Or, it may be based on an object storage service.

[0075] For example, a relational database is used to store structured query and management data. Key database tables include, but are not limited to: a node table, a survey question table, a knowledge unit table, and a tag table. The node table includes, but is not limited to: node identifier, parent node identifier, node type, and child node identifier. The survey question table includes, but is not limited to: question identifier, belonging node identifier, and survey question text. The knowledge unit table includes, but is not limited to: knowledge unit identifier, belonging node identifier, and knowledge content. The tag table includes, but is not limited to: tag identifier, associated node identifier, and tag text. The relational database can be a PostgreSQL database or a MySQL database.

[0076] For example, NoSQL databases are used to store document-based data, where each document represents a node and includes research questions, knowledge units, tags, parent node identifiers, child node identifiers, etc. NoSQL databases can be MongoDB or Cassandra.

[0077] For example, a graph database is used to represent complex relationships between nodes and supports complex relationship queries. Each graph node corresponds to a node in a tree structure, and edges represent the relationships between nodes.

[0078] For example, object storage services are used to store tree-structured knowledge bases. Object storage services can be Amazon Web Services (AWS) Simple Storage Service (S3) storage services.

[0079] It should be noted that the data input for the pre-defined knowledge base uses a combination of automated scripts and manual methods to ensure data accuracy and timely updates. Subsequently, data maintenance tools allow users to add, modify, or delete data stored in the knowledge base.

[0080] This allows for flexible processing and querying of data of different scales, while ensuring continuous data input and regular system maintenance.

[0081] After introducing the pre-set knowledge base, the method for generating survey questions will be introduced below.

[0082] As shown in Figure 2, Figure 2 illustrates a method for generating survey questions according to some embodiments, the method including: S201 to S203.

[0083] S201, The first question is raised.

[0084] The first question is one of several pre-set questions.

[0085] In one design, preset questions include open-ended preset questions and closed-ended preset questions. Open-ended preset questions are those without a fixed answer, while closed-ended preset questions are those with a fixed answer.

[0086] In one design, multiple preset questions can be stored in a preset knowledge base. These preset questions in the knowledge base can be divided into multiple levels, with the first question located at the first level of these levels.

[0087] It should be noted that, starting from the highest level and proceeding downwards, the preset questions for each level are the preset questions that are further subdivided from the preset questions of the previous level.

[0088] For example, the first preset question for level a is "How is your company currently using its energy management system?" The second preset question for level b is "Based on the use of the energy management system, how does your company optimize peak-valley-flat electricity usage to reduce electricity costs?" Level a is the level above level b, and the second preset question is a further subdivision of the first preset question.

[0089] It should be noted that this disclosure does not limit the data structure of the preset knowledge base. For example, the preset knowledge base may be a tree-like data structure. Or, for example, the preset knowledge base may be a graph-like data structure.

[0090] In this embodiment of the disclosure, each level of the preset knowledge base has at least one preset node. Each preset node includes at least one of the following: a node identifier, a category label, a knowledge unit, and a preset question. The node identifier is used to indicate the path of the preset node, and the knowledge unit is used to indicate the answer to the preset question.

[0091] It should be noted that the highest-level preset node is the root node, and the lowest-level preset node is the leaf node. Starting from the highest level, the preset questions are progressively lowered, with each level's preset question being a refinement of the question from the previous level. The root node does not have category labels or knowledge units, and the leaf nodes do not have preset questions.

[0092] For example, as shown in Figure 3, the preset nodes include a root node, intermediate nodes, and leaf nodes. The root node includes a node identifier and a preset question. The intermediate nodes include a node identifier, a category label, a knowledge unit, and a preset question. The leaf nodes include a node identifier, a category label, and a knowledge unit.

[0093] In one implementation, the survey question generation device pre-stores a preset knowledge base. In response to a user's question triggering action, a first question is determined from the preset knowledge base. Then, the first question is issued.

[0094] In one design, the first problem can be a preset problem in the root node. Alternatively, the first problem can be a preset problem in an intermediate node.

[0095] S202. Obtain the answer to the first question.

[0096] It should be noted that this disclosure does not limit the data type of the answer to the first question. For example, the answer to the first question can be audio data or text data.

[0097] In one implementation, the answer to the first question sent by the user equipment can be received.

[0098] In another implementation, the user's voice data can be obtained to get the answer to the first question.

[0099] In another implementation, the survey question generation device pre-stores multiple question answers, with one answer corresponding to one question. Based on the first question, the answer to the first question is obtained.

[0100] S203. Based on the answer to the first question, determine the second question from multiple preset questions.

[0101] The second question is one of several pre-set questions, and it is related to the first question.

[0102] In this embodiment of the disclosure, the second problem is located at the second level. The association between the second problem and the first problem may include: the first level and the second level being the same level, or the second level being the next level after the first level.

[0103] For example, referring to Figure 4, the second level is the level below the first level. The preset node a containing the first question is located in the first level, and the preset node b containing the second question is located in the second level. The node identifier of preset node a is 1-1, and the node identifier of preset node b is 2-1. Alternatively, the preset node b containing the first question is located in the first level, and the preset node c containing the second question is located in the second level. The node identifier of preset node b is 2-1, and the node identifier of preset node c is N-1. The first and second levels are the same level. The preset node b containing the first question is located in the first level, and the preset node d containing the second question is located in the second level. The node identifier of preset node b is 2-1, and the node identifier of preset node d is 2-2. The preset question for preset node a is "What challenges has your company encountered in the process of digital and intelligent transformation?" Preset node b is categorized under "Energy Management," "Energy Efficiency," and "Energy Optimization." Its preset question is "How is your company currently using its energy management system?" Its knowledge unit is "For textile companies, effective energy management is key to improving energy efficiency, reducing costs, and supporting environmental sustainability...". Preset node c is categorized under "Air Compressor," "Compressor," and "Compressed Air System." Its preset question is "Based on the use of an energy management system, how does your company optimize peak-valley-level electricity usage to reduce electricity costs?" Its knowledge unit is "For textile companies, air compressors are one of the main energy-consuming devices...". Preset node d is categorized under "Production Efficiency," "Technology Integration," and "Division of Responsibilities." Its preset question is "What is the detailed implementation status of technology integration?" Its knowledge unit is "In the digital and intelligent transformation of textile companies, it is necessary to clarify the goals of technology integration...".

[0104] In one implementation, based on the answer to the first question, an optimized large language model is used to determine the target node in the second level from a pre-defined knowledge base. The large language model is then used to generate a new question based on the answer, where the category label in the target node matches the answer to the first question. Subsequently, based on the target node, the second question is obtained.

[0105] In one design, multiple pre-defined models can be obtained. These models are then evaluated and tested to obtain an initial large language model. Next, the initial large language model is fine-tuned to obtain a fine-tuned large language model. This fine-tuned model is then validated to obtain validation results. Finally, based on the validation results, the fine-tuned large language model is optimized, and the optimized model is used to determine the target nodes in the second level from the pre-defined knowledge base.

[0106] For example, as shown in Figure 5, which is a flowchart illustrating another method for generating survey questions according to some embodiments, multiple models are evaluated based on their performance on information retrieval tasks to obtain an initial model. Then, the initial model is tested using a training dataset, and its capabilities and efficiency in handling specific tasks are analyzed, along with the availability of computational resources, to obtain an initial large language model. Next, the target dataset is loaded and preprocessed. For example, data annotation is performed on the target dataset, and the input and output of the initial large language model are concatenated in a specific format to clearly identify different parts, such as input and output. Then, the large language model and word segmenter are loaded, and the training components of the large language model are defined for fine-tuning. Training components include, but are not limited to, a trainer, training hyperparameters, and a data warper. Training hyperparameters can be batch size or learning rate. Finally, the fine-tuned large language model is validated using quantitative metrics to evaluate its overall performance. Quantitative metrics include, but are not limited to, question-answering accuracy and text generation relevance. Subsequently, based on the experimental results, different optimization strategies were adopted to adjust the configuration of the large language model in order to optimize the model parameters, and the large language model with the best performance was selected as the final large language model.

[0107] This allows the large language model to continuously collect new data and feedback in practical applications, further optimizing its matching and response accuracy. It also enhances the model's real-time update capability, ensuring the knowledge base is constantly updated and significantly improving the timeliness and accuracy of research.

[0108] It should be noted that this disclosure does not limit the methods used for fine-tuning. For example, the model can be fine-tuned using low-rank adaptation (LoRA). Another example is prompt tuning (P-tuning).

[0109] It should be noted that LoRA technology can introduce low-rank matrices to adjust the network weights, and P-tuning can train a small number of learnable continuous vectors to guide the model in generating specific outputs. These methods not only maintain the powerful capabilities of the base model but also increase the model's sensitivity and adaptability to the specific needs of the textile industry.

[0110] In some embodiments, after obtaining the answer to the first question, in response to the user's stop survey operation, the target node in the second level is determined from the preset knowledge base using an optimized large language model. Then, based on the target node, the knowledge unit of the target node is obtained. Finally, the knowledge unit of the target node containing the second question is output, and the dialogue ends.

[0111] It should be noted that the knowledge unit of the target node where the second problem is located is the solution to the first problem.

[0112] In this way, the research can conclude at any stage, and it ensures that a corresponding knowledge unit exists for each research question to arrive at a solution to the first question. Furthermore, the solution to the first question can be linked to the answer to the first question, enabling precise problem identification and providing more targeted and practical solutions for the respondents.

[0113] Based on the above technical solution, a first question can be posed, which is one of several pre-set questions. Then, the answer to the first question is obtained. Based on the answer to the first question, a second question can be determined from the multiple pre-set questions, and the second question is related to the first question. This strengthens the correlation between the answers to the second and first questions, allows for understanding the implicit intentions of the responses, and makes the research questions more precise, thereby improving the efficiency and quality of the dialogue and making the research results more accurate.

[0114] As shown in Figure 6, Figure 6 illustrates another method for generating survey questions according to some embodiments. In this method, S203 may include S601 to S602.

[0115] S601. Based on the answer to the first question and the first level, determine the target label.

[0116] Each preset question corresponds to at least one category label. The category label is used to reflect the keywords of the preset question.

[0117] In one implementation, the answer to the first question is parsed using a large language model to determine at least one candidate label. Based on this first level, the target label is determined from the at least one candidate label.

[0118] In this embodiment of the disclosure, target tags can be determined based on a problem survey model.

[0119] In one design, if the problem survey mode is a deep survey mode, the target label is determined based on the category labels in the next level after the first level and at least one candidate label. If the problem survey mode is a broad survey mode, the target label is determined based on the category labels in the first level and at least one candidate label.

[0120] It should be noted that the method for determining the target label based on the category label in the first level and at least one candidate label is not limited. For example, if a candidate label is identical to the category label corresponding to the preset question, then that candidate label is determined to be the target label. As another example, if the similarity between a candidate label and the category label corresponding to the preset question exceeds a preset similarity threshold, then that candidate label is determined to be the target label.

[0121] In some embodiments, when the question survey mode is a breadth survey mode and the second question is not derived based on the answer to the first question, the question survey mode can be changed to a depth survey mode.

[0122] In other words, in-depth research mode and breadth research mode can be switched. This makes the research process more efficient and adaptable, improves the response speed of the research, and enhances the comprehensiveness and depth of the research results.

[0123] In one design, when the problem survey mode is a breadth survey mode, if at least one of the candidate labels is the same as the category label in the first level, then the same label is used as the target label. If at least one of the candidate labels is not the same as the category label in the first level, then the target label is determined based on the category label in the next level after the first level and at least one candidate label.

[0124] S602. Based on the target label, determine the second problem.

[0125] In one implementation, the target node containing the second question can be determined using a large language model based on the target label. Then, the second question can be retrieved from the target node.

[0126] In some embodiments, if the problem survey mode is an in-depth survey mode and the first level is not the last level among multiple levels, then a second question is determined from multiple preset questions based on the answer to the first question.

[0127] Based on the above technical solution, target tags can be determined based on the answer to the first question and the first level. Then, based on the target tags, the second question can be determined. This allows for faster identification of the question most closely related to the answer. Furthermore, as the levels deepen, the questions become increasingly detailed, helping users to delve deeper into detailed information and details, thereby improving the accuracy and depth of the research.

[0128] The embodiments of this disclosure will now be described in detail with reference to specific examples.

[0129] Establish a pre-defined knowledge base for textile enterprises and train a large-scale model for the textile industry to generate research questions, including the following sub-steps:

[0130] Step 1: Establish a knowledge base for research and diagnosis of textile enterprises.

[0131] Data sets were collected from the textile industry. The professional data and knowledge were then categorized into six main areas: textile product knowledge, textile industry knowledge, textile technology knowledge, textile experience knowledge, textile equipment knowledge, and training management materials, to ensure comprehensive data and knowledge collection. Textile product knowledge includes at least one of the following: textile specification sheets, textile user and maintenance manuals, fiber and fabric performance parameter tables, textile quality certificates, textile samples and design catalogs, and textile quality test reports. Textile industry knowledge includes at least one of the following: textile industry development trend reports, textile market research and demand analysis reports, textile industry standards and related regulations, textile industry news and latest developments, reports on leading textile enterprises, and a dictionary of textile terminology and definitions. Textile technology knowledge includes at least one of the following: textile production process flow documents, textile machinery operation manuals, textile technical parameters and performance manuals, textile quality control standards and specifications, introductions to new textile materials and processes, and compilations of textile-related technology patents. Textile experience knowledge includes at least one of the following: textile project case studies, textile project execution summary reports, textile production problem solutions, textile industry best practice guidelines, textile employee experience sharing documents, and summaries of customer and market feedback. Textile equipment knowledge includes at least one of the following: textile machinery operation manual, textile equipment repair and maintenance guide, introduction to textile automation technology, textile machinery efficiency optimization strategies, introduction to new textile machinery and equipment, and safe operating procedures for textile equipment. Training management materials include at least one of the following: textile industry new employee induction training manual, textile skills training materials, textile safety production training materials, textile management capacity building tutorials, textile product knowledge training materials, and lecture materials on textile industry development trends.

[0132] Next, necessary information was collected through interviews and questionnaires. First, the research objectives and expected outcomes were clarified. Second, expert selection criteria were determined, such as experience, position, and professional field. Third, research methods were selected, such as in-depth interviews, focus groups, and questionnaires. Fourth, interview outlines or questionnaires were prepared. Fifth, supporting materials were prepared, such as interview consent forms and data collection forms. Finally, a report was written and knowledge was integrated.

[0133] Step 2: Organize and structure textile-related knowledge.

[0134] First, select parsing tools and libraries that are compatible with the format of the documents related to textile knowledge. Document content typically includes three types: text, images, and tables. Text types include PDF, Word, Excel, and CSV documents. For example, for PDF documents, you can use the PyPDF2 or PDFMiner library for parsing; for Word documents, you can use the python-docx library; for Excel documents, you can use the openpyxl or pandas library; and for CSV documents, you can use Python's built-in csv library. For tabular data, you can use the Camelot or Tabula-py library combined with pandas for effective recognition and extraction. For image documents, you can use PDFMiner and Adobe Acrobat image extraction services, combined with OCR and multimodal large model related technologies, to convert the text information in the images.

[0135] Next, the extracted text information is preprocessed. For example, tools such as NLTK, spaCy, TextBlob, Gensim, and scikit-learn can be used for text preprocessing. Preprocessing includes, but is not limited to, removing redundant information, stop words, and special characters.

[0136] Subsequently, the preprocessed text information can be divided into manageable knowledge units using various strategies. For example, it can be divided into manageable knowledge units based on a fixed size, such as characters, words, or token counts. Another example is the iterative division based on predefined delimiters, such as the newline character '\n'. Yet another example is the division based on semantic similarity, aggregating content with similar topics or concepts. Still another example is the division based on document structure, such as titles and chapters. Then, tools like Gensim, NLTK, Sentence Transformers, and large-scale language models are used to help identify and label key concepts, thereby extracting key knowledge points such as "capacity utilization rate," "market demand," and "energy consumption."

[0137] Step 3: Organize knowledge hierarchically based on a tree structure.

[0138] Construct a tree-structured management system for research questions and knowledge units. Then, use the root node of the tree as the initial question. For example, "What challenges has your company encountered in its digital and intelligent transformation?" The root node is linked to multiple intermediate nodes. These intermediate nodes include node identifiers, category labels, information units, and detailed questions. For example, the category label might be "Energy Management," and the detailed question might be "Which production processes have already applied digital technologies?" Intermediate nodes can link to the next level of intermediate node identifiers, or they can link to the bottom-level leaf node identifiers. The bottom-level leaf nodes include node identifiers, labels, and information units.

[0139] Next, based on the pre-defined problem, the extracted tags and information units are assigned to the corresponding hierarchical nodes in the tree structure. Then, a unique node identifier is defined for each node, clearly depicting its subordinate paths. For example, this can be combined with the layer number and the node number within that layer. For instance, assuming "1-1" is the root node, the first and second nodes of the second layer can be labeled "2-1" and "2-2" respectively. The corresponding subordinate paths can be expressed using node identifiers, such as "1-1 / 2-1 / 3-1". Therefore, appropriate separators need to be pre-selected to distinguish the nodes in the path. For example, " / " or "->".

[0140] Step 4: Design and implement a dedicated data structure to store and manage the pre-defined knowledge base.

[0141] Knowledge bases can be implemented using relational databases, NoSQL databases, graph databases, and object storage services. Relational databases can be PostgreSQL or MySQL, NoSQL databases can be MongoDB or Cassandra, and object storage services can be AWS S3. If using a relational database, data can be organized using structures such as node tables, question tables, knowledge unit tables, and tag tables.

[0142] Once the database framework is built, a combination of automated scripts and manual operations can be used to populate, update, and edit knowledge. For example, scripts can be pre-set to input preset questions into the knowledge base and automatically establish relationships with other relevant data tables. Afterward, regular database maintenance and update processes can be set up to ensure the accuracy and timeliness of information.

[0143] Step 5: Evaluation and selection of large language models.

[0144] Multiple large language models can be evaluated using GLUE, SuperGlue, and MMLU to obtain evaluation results. Based on these results, the model with the best performance is selected to meet the detailed needs of the textile industry. Subsequently, by utilizing collected textile industry information resources, the selected model can be further customized and optimized to improve its understanding of industry terminology, efficiency in processing industry data, and performance in practical applications. For example, industry reports, market analysis, and production data can be used to optimize the large language model to improve production efficiency, optimize supply chain management, and enhance product quality control.

[0145] Step six: Fine-tune the model.

[0146] Next, a large-scale language model specifically designed for the textile industry is constructed, and the collected industry information is appropriately formatted according to the characteristics of the chosen base model. For example, the data is organized into the format required for model fine-tuning, and key information such as inputs and outputs is clearly labeled. Alternatively, data templates can be created to standardize the structure of training samples, ensuring that all important variables and parameters are clearly labeled and recorded. Then, supervised training methods are used to fine-tune the model. For example, LoRA or P-tuning can be used for model fine-tuning.

[0147] Step 7: Verification and optimization of the large-scale model for the textile industry.

[0148] Next, a comprehensive evaluation of the fine-tuned model is conducted using quantitative standards to obtain the evaluation results. For example, the BLEU metric can be used to evaluate the language quality of the model when generating specific text. Alternatively, F1 scores and accuracy can be used to quantify the model's performance when processing fixed data slots. Furthermore, human testing can be incorporated to evaluate the model's performance in real-world applications, ensuring that the model's output meets the actual needs and context of the textile industry.

[0149] Subsequently, based on the evaluation results, the model is further optimized using different techniques and strategies. For example, incremental learning can be used to gradually introduce new data. Another example is adversarial training methods, which can enhance the model's robustness to anomalous inputs. Yet another example is hyperparameter tuning, optimizing parameters such as the learning rate and batch size during training to improve model performance.

[0150] Step 8: Issue the pre-set question and collect the answers.

[0151] Next, as shown in Figure 7, which is a flowchart illustrating another method for generating survey questions according to some embodiments, the system issues a first question and obtains its corresponding first question answer. Then, based on the first question answer, it searches for the next question to obtain the second question. Starting from the root node of the tree structure, the questions at each node are issued in voice or text form. Next, the system collects the question answers and automatically converts the information of these questions and answers according to a predefined format of the textile industry's large-scale model to ensure the structure and standardization of the information. For example, a question answer might trigger the system to query related intermediate nodes, which may involve specific production problems or technological applications. The system analyzes the question answers and matches them with nodes in the knowledge base.

[0152] Step nine: Obtain the child node most relevant to the answer to the question.

[0153] Next, based on the question and answer of the current node, operations are performed. The category labels of child nodes are analyzed using a large-scale textile industry model, and combined with pre-defined prompts, the top k child nodes most relevant to the question and answer are obtained. Then, information such as the child node's identifier, knowledge unit, and research question can be stored using a pre-defined data structure. This pre-defined data structure can be a hash table, a heap structure, or a stack structure.

[0154] Furthermore, the preset knowledge base allows users to select traversal modes to generate research questions. For example, users can choose between breadth-first search or depth-first search. If a user needs to quickly obtain research questions, a breadth-first strategy can be used; if a user needs to gain in-depth understanding of the textile industry, a depth-first strategy can be used. This process iterates until the user's query requirements are met or the end of the knowledge base is reached.

[0155] The foregoing primarily describes the solutions provided by the embodiments of this disclosure from a methodological perspective. It is understood that the survey question generation apparatus, in order to achieve the aforementioned functions, includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the survey question generation method steps described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0156] This disclosure also provides a survey question generation apparatus. This survey question generation apparatus can be a server, a CPU within the server, a generation module within the server for generating survey questions, or a client within the server for generating survey questions.

[0157] This disclosure embodiment can divide the survey question generation device into functional modules or functional units according to the above method example. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The division of modules or units in this disclosure embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0158] This disclosure provides a survey question generation apparatus. As shown in FIG8, the survey question generation apparatus may include an acquisition module 801 and a processing module 802.

[0159] Processing module 802 is used to issue a first question, which is one of a plurality of preset questions. Acquisition module 801 is used to acquire the answer to the first question. Processing module 802 is also used to determine a second question from the plurality of preset questions based on the answer to the first question, the second question being related to the first question.

[0160] Figure 9 is a schematic diagram of another survey question generation apparatus according to some embodiments. This survey question generation apparatus may include a processor 902, which executes application code to implement the survey question generation method of this disclosure.

[0161] The processor 902 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs according to the present disclosure.

[0162] As shown in Figure 9, the apparatus for generating the survey questions may further include a memory 903. The memory 903 is used to store the application code that executes the present invention, and its execution is controlled by the processor 902.

[0163] Memory 903 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 903 may exist independently and be connected to processor 902 via bus 904. Memory 903 may also be integrated with processor 902.

[0164] As shown in Figure 9, the survey question generation device may further include a communication interface 901. The communication interface 901, processor 902, and memory 903 can be coupled to each other, for example, through a bus 904. The communication interface 901 is used for information exchange with other devices, for example, supporting information exchange between the survey question generation device and other devices.

[0165] It should be noted that the device structure shown in Figure 9 does not constitute a limitation on the device for generating the survey question. In addition to the components shown in Figure 9, the device for generating the survey question may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0166] In actual implementation, the functions implemented by the processing unit can be achieved by the processor 902 shown in Figure 9 calling the program code in the memory 903.

[0167] This disclosure also provides a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing instructions that, when executed by a processor of a computer device, enable the computer to perform the research problem generation method provided in the embodiments described above. For example, the computer-readable storage medium may be a memory 903 including instructions that can be executed by a processor 902 of a computer device to complete the method. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0168] Figure 10 is a conceptual partial view of a computer program product according to some embodiments, the computer program product including a computer program for executing computer processes on a computing device.

[0169] In one embodiment, the computer program product is provided using a signal carrying medium 1000. The signal carrying medium 1000 may include one or more program instructions that, when executed by one or more processors, can provide the functions or parts thereof described above with reference to Figures 2 and 6. Therefore, for example, referring to the embodiment shown in Figure 2, one or more features of S201 to S203 may be borne by one or more instructions associated with the signal carrying medium 1000. Furthermore, example instructions are also described in the program instructions of Figure 10.

[0170] In some examples, the signal carrying medium 1000 may include a computer-readable medium 1001, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video optical disc (DVD), a digital magnetic tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc.

[0171] In some implementations, the signal carrying medium 1000 may include a computer recordable medium 1002, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, and so on.

[0172] In some implementations, the signal carrying medium 1000 may include a communication medium 1003, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0173] The signal-bearing medium 1000 can be transmitted by a wireless communication medium 1003. One or more program instructions may be, for example, computer-executable instructions or logical implementation instructions.

[0174] In some examples, the apparatus for generating survey questions can be configured to provide various operations, functions, or actions in response to one or more program instructions in a computer-readable medium 1001, a computer-recordable medium 1002, and / or a communication medium 1003.

[0175] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0176] In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the constituent units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0178] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, essentially, or the parts that contribute to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0180] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for generating survey questions, comprising: The first question is posed, which is one of a plurality of pre-set questions; Get the answer to the first question corresponding to the first question; Based on the answer to the first question, a second question is determined from the plurality of preset questions, and the second question is related to the first question.

2. The method of claim 1, wherein, The multiple preset questions are divided into multiple levels, with the first question located in the first level of the multiple levels and the second question located in the second level of the multiple levels; The second problem is related to the first problem in the following ways: the first level and the second level are at the same level, or the second level is the next level after the first level.

3. The method of claim 2, wherein, One of the preset questions corresponds to at least one category label; determining the second question from the plurality of preset questions based on the answer to the first question includes: Based on the answer to the first question and the first level, the target label is determined; Based on the target label, the second problem is determined.

4. The method of claim 3, wherein, Determining the target label based on the answer to the first question and the first level includes: The answer to the first question is parsed using a large language model to determine at least one candidate label. The large language model is then used to generate a new question based on the answer. Based on the first level, the target label is determined from the at least one candidate label.

5. The method according to claim 4, further comprising: Obtain the problem investigation model; The step of determining the target label from the at least one candidate label based on the first level includes: If the problem survey mode is an in-depth survey mode, then the target label is determined based on the category label in the next level of the first level and the at least one candidate label; If the problem survey mode is a breadth survey mode, then the target label is determined based on the category labels in the first level and the at least one candidate label.

6. The method of claim 5, wherein, If the problem survey mode is a breadth survey mode, then based on the category labels in the first level and the at least one candidate label, the target label is determined, including: If any of the at least one candidate label has the same category label as the one in the first level, then the same label is taken as the target label; If none of the at least one candidate label is the same as the category label in the first level, then the target label is determined based on the category label in the next level after the first level and the at least one candidate label.

7. The method of claim 2, wherein, The step of determining the second question from the plurality of preset questions based on the answer to the first question includes: If the problem survey mode is an in-depth survey mode, and the first level is not the last level among the multiple levels, then the second question is determined from the multiple preset questions based on the answer to the first question.

8. A device for generating survey questions, comprising an acquisition module and a processing module: The processing module is used to issue a first question, which is one of a plurality of preset questions; The acquisition module is used to acquire the answer to the first question corresponding to the first question; The processing module is further configured to determine a second question from the plurality of preset questions based on the answer to the first question, wherein the second question is related to the first question.

9. An apparatus for generating survey questions, comprising: Processor and memory; The processor and the memory are coupled; The memory is used to store one or more programs, the one or more programs including computer execution instructions. When the survey question generation device is running, the processor executes the computer execution instructions stored in the memory to cause the survey question generation device to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium having stored therein instructions, wherein, When the computer executes the instructions, the computer performs the method according to any one of claims 1-7.

11. A computer program product comprising instructions, wherein, When the instruction is executed by the computing device, the computing device performs the method according to any one of claims 1-7.

Images