Text analysis problem generation method and device, electronic equipment and storage medium

By acquiring the identity role of the large language model and obtaining professional knowledge based on self-question and answering, and generating text analysis problems, the problems of low efficiency and high cost of text analysis problems in the existing technology are solved, and high-quality problem generation and efficiency improvement are achieved.

CN119066164APending Publication Date: 2024-12-03BEIJING QIYI CENTURY SCI & TECH CO LTD

Patent Information

Application Number
CN202411014548.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing large language models require long-term training in text analysis, high hardware resource costs and lack standard answers, resulting in low efficiency and high cost in generating text analysis problems, unable to easily obtain training sets, and a large amount of manpower investment is required.

Method used

By acquiring the identity roles of the large language model, obtaining professional knowledge based on self-question and answering, generating text analysis questions, reducing the need for model fine-tuning, improving problem generation efficiency, and reducing costs.

Benefits of technology

The generation of high-quality text analysis problems is realized, which reduces the generation time, avoids a large amount of manpower investment, reduces costs, and improves the generation efficiency of text analysis problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066164A_ABST
    Figure CN119066164A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a text analysis problem generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an identity role of a large language model, and enabling the identity role to correspond to the text type of a to-be-analyzed text; according to the identity role, obtaining professional knowledge corresponding to the text type through the large language model based on a self-question-and-answer form; according to the professional knowledge, a text analysis problem for analyzing the to-be-analyzed text is generated through the large language model on the basis of a self-question-and-answer form, and the text analysis problem is used for asking questions to the large language model so as to analyze the to-be-analyzed text. According to the embodiment of the invention, the generation efficiency of the text analysis problem can be improved, a large amount of manpower does not need to be invested, and the cost can be reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, electronic device and storage medium for generating text analysis questions. Background Art

[0002] The emergence of Large Language Model (LLM) has made it possible to analyze complex long texts, such as the deconstruction, analysis, and evaluation of plots, scripts, and literary products, but there is still no complete general and vertical solution. By using LLM to analyze long texts, it is necessary to be able to ask high-quality questions about specific long texts.

[0003] A model that can solve long text analysis can be obtained by fine-tuning the LLM pre-trained model. However, fine-tuning the LLM pre-trained model takes a long time to train and has high hardware resource costs. In addition, since there is no standard answer to such problems, training sets cannot be easily obtained. If a certain amount of training data is to be formed, a large amount of manpower with corresponding professional background is required, which is costly. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and storage medium for generating text analysis questions, so as to improve the efficiency of generating text analysis questions and reduce the cost. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a method for generating a text analysis question is provided, comprising:

[0006] Obtaining an identity role of a large language model, wherein the identity role corresponds to a text type of a text to be analyzed;

[0007] According to the identity role, the professional knowledge corresponding to the text type is acquired through the large language model in a self-question-answering format;

[0008] According to the professional knowledge, text analysis questions for analyzing the text to be analyzed are generated by the large language model in a self-question-answering format, and the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

[0009] In a second aspect of the present invention, there is also provided a device for generating text analysis questions, comprising:

[0010] A role acquisition module, used to acquire the identity role of the large language model, wherein the identity role corresponds to the text type of the text to be analyzed;

[0011] A professional knowledge acquisition module, used to acquire professional knowledge corresponding to the text type based on the identity role through the large language model in a self-question-answering format;

[0012] An analysis question generation module is used to generate text analysis questions for analyzing the text to be analyzed based on the professional knowledge and in a self-question-answering format through the large language model, wherein the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

[0013] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned methods for generating text analysis problems.

[0014] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned methods for generating text analysis questions.

[0015] The method, device, electronic device and storage medium for generating text analysis questions provided by the embodiments of the present invention obtain the identity role of a large language model, and based on the identity role, obtain the professional knowledge that the identity role should have through the large language model in the form of self-questioning and answering, and generate a text analysis text for analyzing the text to be analyzed through the large language model in the form of self-questioning and answering according to the professional knowledge. Since the professional knowledge that the identity role should have can be obtained based on the identity role through the large language model, high-quality text analysis questions can be generated based on the professional knowledge, and no fine-tuning training is required, which can reduce the generation time of text analysis questions and improve the generation efficiency of text analysis questions. Moreover, it does not require a large amount of manpower, which can reduce costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0017] Figure 1 is a flow chart of a method for generating text analysis questions provided by an embodiment of the present invention;

[0018] Figure 2 is a flow chart of acquiring professional knowledge based on a self-question-and-answer format in an embodiment of the present invention;

[0019] Figure 3 is an example diagram of a first example process in an embodiment of the present invention;

[0020] Figure 4 is a flowchart of generating text analysis questions based on the self-question-answering format in an embodiment of the present invention;

[0021] Figure 5 is a schematic diagram of a text analysis framework in an embodiment of the present invention;

[0022] Figure 6 It is a structural schematic diagram of a device for generating text analysis questions provided by an embodiment of the present invention;

[0023] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0025] Figure 1 is a flowchart of a method for generating a text analysis question provided by an embodiment of the present invention. Figure 1 As shown, the method for generating the text analysis text includes the following steps:

[0026] Step 110 , obtaining the identity role of the large language model, wherein the identity role corresponds to the text type of the text to be analyzed.

[0027] The identity role is a professional identity used to analyze the text to be analyzed of the text type. For example, when the text type is a movie-related text, the identity role is a film critic; when the text type is a script, the identity role is a script analyst; when the text type is a literary work, the identity role is a literary critic, and so on.

[0028] The identity role of the large language model when analyzing the text to be analyzed can be set based on the prompt words, so that the large language model knows the identity role to be played. For example, using the large language model of OpenAI, the identity role of the large language model can be set through the initial system role; or, the identity role of the large language model can be dynamically specified through human-computer interaction.

[0029] Step 120 , according to the identity role, obtain the professional knowledge corresponding to the text type through the large language model in a self-questioning and answering format.

[0030] The first prompt text of chain thinking can be pre-set. The first prompt text is used to prompt the chain thinking process of acquiring professional knowledge. Based on the first prompt text of chain thinking, the large language model learns the process of chain thinking, so that the large language model can acquire the professional knowledge corresponding to the text type based on the identity role and the self-questioning form. The professional knowledge is the background knowledge or professional knowledge that the identity role needs to master. The self-questioning form refers to the large language model asking questions, and after asking the questions, the large language model gives the answers to the questions.

[0031] Step 130, based on the professional knowledge, generate text analysis questions for analyzing the text to be analyzed through the large language model in a self-question-answering format, wherein the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

[0032] A second prompt text of chain thinking can be pre-set. The second prompt text is a chain thinking process for generating text analysis questions based on professional knowledge prompts. Based on the second prompt text of chain thinking, the large language model learns the chain thinking process, so that the large language model generates text analysis questions for analyzing the text to be analyzed based on the professional knowledge and in the form of self-questioning. Subsequently, the text analysis questions can be input into the large language model, so that the large language model can analyze the text to be analyzed based on the text analysis questions.

[0033] The following takes script analysis as an example to illustrate the generation process of text analysis problems: the text type of the text to be analyzed is a script, and the script to be analyzed is input into the large language model in advance; first, the identity role of the large language model is set as a script analyst; according to this identity role, the large language model obtains the professional knowledge required for script analysis in the form of self-questioning and answering. For example, the professional knowledge obtained is "to analyze the script, you need to understand the dramatic conflicts of the characters"; based on this professional knowledge, the large language model obtains the question "Is the dramatic conflict of the characters sufficient?" based on the script to be analyzed, and the large language model answers this question with the answer "The dramatic conflict of the characters is sufficient, there are... (list the dramatic conflicts of the characters) The large language model then asks a further question based on the answer, “For XXX, what are the stories that develop with other characters?” The large language model then obtains the corresponding answer and asks further questions. It continues to ask questions and obtain answers through this chain thinking process until a high-quality text analysis question is obtained. For example, the final text analysis question is “What are the characters that have comedic conflicts with XXX?” This text analysis question is a question asked to the large language model. After obtaining the text analysis question, the large language model can further analyze the script and obtain the answer to the question.

[0034] The method for generating text analysis questions provided in the present embodiment obtains the identity role of a large language model, and based on the identity role, obtains the professional knowledge that the identity role should have through the large language model in the form of self-questioning and answering, and based on the professional knowledge, generates a text analysis text for analyzing the text to be analyzed through the large language model in the form of self-questioning and answering. Since the professional knowledge that the identity role should have can be obtained based on the identity role through the large language model, high-quality text analysis questions can be generated based on the professional knowledge, and no fine-tuning training is required, which can reduce the generation time of text analysis questions, improve the generation efficiency of text analysis questions, and do not require a large amount of manpower, which can reduce costs.

[0035] Figure 2 is a flow chart of acquiring professional knowledge based on self-questioning and answering in an embodiment of the present invention, such as Figure 2 As shown, according to the identity role, the professional knowledge corresponding to the text type is obtained through the large language model in a self-question-answering form, that is, the above-mentioned step 120, including:

[0036] Step 121 : inputting a first prompt text including at least one first process example into the large language model, where the first process example is an example process for acquiring corresponding example expertise based on an example role.

[0037] When acquiring expertise based on identity roles, the Self-Ask (SA) paradigm is adopted to enable the large language model to execute the process of chain thinking. Chain thinking can be expressed as a question, multiple further actions to be taken (follow up), and multiple intermediate answers. The first prompt text includes an example of the first process based on the chain thinking.

[0038] Step 122: Generate professional knowledge questions corresponding to the identity role through the large language model according to the first prompt text.

[0039] The large language model learns the first process example in the first prompt text, clarifies its own identity role, and generates professional knowledge questions corresponding to the identity role. For example, if the text type is a script and the identity role is a script analyst, the generated professional knowledge questions may be "analyze the script for me" or "what are the criteria for being a script analyst".

[0040] Figure 3 is an example diagram of the first example process in an embodiment of the present invention, such as Figure 3As shown, the professional knowledge question generated by the large language model is "Analyze the script for me". When it is necessary to set a role, the further action taken is "Please set my role". The user can enter the identity role of the large language model as "script analyst"; the large language model first defines the identity role as "script analyst" and determines that the further action taken is the professional knowledge question raised, "What are the criteria for being a script analyst."

[0041] Step 123: Acquire a first intermediate answer corresponding to the professional knowledge question through the large language model.

[0042] The action space (Action Space) of the large language model can be pre-set, and any tools that can assist in improving the effect can be added to the action space, for example, it can include the calling interface of the search engine.

[0043] According to professional knowledge questions, the large language model can determine the first intermediate answer based on its own knowledge reserves, or it can obtain the first intermediate answer with the help of search engine tools based on the guidance of the first prompt text.

[0044] In one embodiment of the present invention, obtaining the first intermediate answer corresponding to the professional knowledge question through the large language model includes: inputting the professional knowledge question into the large language model, and receiving a search instruction output by the large language model; searching through a search engine according to search keywords in the search instruction, and parsing the search results to obtain the first intermediate answer.

[0045] Input professional knowledge questions into the big language model. After being processed by the big language model, when it is necessary to call a search engine to obtain an answer, the big language model will output a search instruction, which includes search keywords and a search engine call interface. The program calls the search engine call interface to input search keywords, searches through the search engine, parses and organizes the search results, obtains a first intermediate answer, and returns the first intermediate answer to the big language model. Figure 3 As shown, by calling the search engine, the keyword retrieval question is "What are the criteria for being a script analyst", and the retrieval result (response Response) obtained is "from characters, plot, rhythm, conflict...", and the retrieval result is used as the first intermediate answer and input into the large language model.

[0046] By integrating the search engine functions and tool introductions into the prompts and combining them with paradigm content design, professional information can be supplemented by calling the search engine interface.

[0047] Step 124, reflect on the first intermediate answer through the large language model and determine the first operation for the next step.

[0048] The ReAct paradigm allows the large language model to both think and act. The ReAct paradigm includes a triplet of reflection on the result (thought), further action to be taken (action), and observation of the effect after the action (observation). The ReAct paradigm allows the large language model to alternately generate reasoning trajectories and text actions. Among them, the process of reflection by the large language model is the process of generating reasoning trajectories by simulating the human thinking process. The reasoning trajectory does not affect the external environment, but affects the internal state of the large language model itself, by reasoning about the context (Context) and updating the context with useful information. The text action affects the external environment and obtains observation feedback (Observation) from the external environment to update the context.

[0049] Based on the guidance of the first prompt text, the first intermediate answer is reflected upon through the large language model, and the first operation for the next step (i.e., further action to be taken) is determined.

[0050] Step 125: Execute the first operation through the large language model to obtain a first operation result, and observe the first operation result.

[0051] Based on the guidance of the first prompt text, the first operation is performed through the large language model to obtain the first operation result. The first operation can be to search for relevant knowledge content with the help of a search engine. After obtaining the first operation result, the large language model can observe the effect of the first operation result. After obtaining the first operation result, it can also be provided to the user for viewing through human-computer interaction. Among them, observation is like the five senses of the large language model, synchronizing external feedback information to the large language model, and assisting the large language model in further analysis or decision-making. In an embodiment of the present invention, the large language model observes the operation result means that the large language model obtains the effect corresponding to the operation result, and determines whether the professional knowledge corresponding to the text type is obtained.

[0052] Step 126, determining whether the professional knowledge corresponding to the text type is obtained based on the observation conclusion, if not, iteratively executing steps 124 to 126 based on the observation conclusion until the professional knowledge corresponding to the text type is obtained.

[0053] The large language model can determine whether sufficient professional knowledge corresponding to the text type has been acquired based on the conclusion of the observation, or it can also determine whether sufficient professional knowledge corresponding to the text type has been acquired through human-computer interaction. If it is determined based on the conclusion of the observation that sufficient professional knowledge corresponding to the text type has not been acquired, then based on the conclusion of the observation, iteratively execute steps 124 to 126 to reflect on the conclusion of the observation through the large language model, and determine the first operation of the next step, execute the first operation through the large language model, obtain the first operation result, and observe the first operation result, and determine whether the professional knowledge corresponding to the text type has been acquired based on the conclusion of the observation, and iteratively execute this process until the professional knowledge corresponding to the text type is acquired.

[0054] like Figure 3 As shown, after the large language model searches through the search engine, it uses the retrieved results as the first intermediate answer and reflects on the first intermediate answer. The reflective question (Reflection 1) is "As I understand the standards, what are the specific contents of these standards?" The first operation (Action 1) is to search through the search engine, and the conclusion of the observation (Observation 1) is "For characters,...; For rhythm,...", that is, observation is a further refinement of the first intermediate answer, and the operations of reflection, action, and observation are iteratively performed until sufficient professional knowledge corresponding to the text type is obtained.

[0055] By inputting the first prompt text into the large language model, the large language model can use the ReAct paradigm to acquire the expertise corresponding to the text type based on the prompt of the first process example, and can acquire sufficient expertise required for text analysis of the text type.

[0056] On the basis of the above technical solution, after acquiring the professional knowledge corresponding to the text type, it also includes: adding the question and answer record of the acquired professional knowledge to the first prompt text in the form of the first process example.

[0057] After obtaining the professional knowledge corresponding to the text type, the professional knowledge questions corresponding to the identity role are generated, the first intermediate answers are obtained, and the question and answer records of obtaining professional knowledge such as reflection and observation are taken as a first process example, and the first process example is added to the first prompt text to better guide the large language model to obtain the corresponding professional knowledge based on the identity role.

[0058] In one embodiment of the present invention, the obtaining of the professional knowledge corresponding to the text type through the large language model in the form of self-questioning and answering according to the identity role includes: before executing the current round of question and answering through the large language model, using the question and answer records of a preset length before the current round as primary memory information, and generating summary information corresponding to the question and answer records before the preset length, and using the summary information as secondary memory information; inputting the current round of questions, the primary memory information and the secondary memory information into the large language model, and executing the current round of question and answering through the large language model.

[0059] In the process of acquiring professional knowledge through a large language model, the contextual history information containing the question and answer records is used as a parameter for requesting a large language model. Before executing the current round of questions and answers through the large language model, the question and answer records of the preset length before the current round and adjacent to the current round are completely retained, and these question and answer records are used as primary memory information. If there is at least one round of question and answer records before the preset length, the content of the question and answer records before the preset length is summarized to generate summary information of the question and answer records before the preset length, and the summary information is used as secondary memory information. The questions, primary memory information and secondary memory information of the current round are input into the large language model, and the large language model determines the answers to the questions of the current round by performing in-context learning on the primary memory information and the secondary memory information to execute the current round of questions and answers. Among them, the preset length is the data length of the question and answer record, for example, it can be represented by a preset round, or it can also be a preset number of characters, a preset number of tokens, etc. The first-level memory information is the memory information adjacent to the current round, and is the complete question and answer record closest to the current round; the second-level memory information is the memory information before the first-level memory information. Since it is farther away from the current round, the summary information of these question and answer records is retained.

[0060] By using the question and answer records of a preset length before the current round as the primary memory information and the summary information corresponding to the question and answer records before the preset length as the secondary memory information, the integrity of the context information input into the large language model can be ensured.

[0061] On the basis of the above technical solution, the method further includes: if the data volume of the question and answer record before the current round is greater than the preset data volume, converting the question and answer record before the current round into a text vector, and storing the text vector in a vector library;

[0062] The step of inputting the current round of questions, the primary memory information and the secondary memory information into the large language model, and executing the current round of question and answer through the large language model includes: when the sum of the lengths of the current round of questions, the primary memory information and the secondary memory information is less than the input window length of the large language model, converting the current round of questions into question vectors, retrieving a target text vector whose similarity to the question vector is greater than a similarity threshold from the vector library, and acquiring text content corresponding to the target text vector; and inputting the current round of questions, the primary memory information, the secondary memory information and the text content into the large language model, and executing the current round of question and answer through the large language model.

[0063] If the number of question and answer records of a large language model is large, or the amount of information in a single round of questions or answers is large, the amount of data in the question and answer records before the current round will be greater than the preset amount of data. In this case, the question and answer records can be included in the vector library as tertiary memory information, which is long-term memory information. When the question and answer records are included in the vector library, the question and answer records before the current round are vectorized (embedding), and the question and answer records before the current round are converted into text vectors, and the text vectors are stored in the vector library as tertiary memory information.

[0064] When executing the current round of question and answer, if the sum of the lengths of the question, the primary memory information and the secondary memory information of the current round is less than the input window length of the large language model, text content with higher similarity can be retrieved from the vector library based on the question of the current round, that is, the question of the current round is converted into a vector form to obtain a question vector, and the similarity between the question vector and the text vector in the vector library is calculated, and a target text vector whose similarity is greater than a similarity threshold is determined, and the text content corresponding to the target text vector is obtained, and the text content is used as context supplementary information; the question of the current round, the primary memory information, the secondary memory information and the obtained text content are input into the large language model, and the current round of question and answer is executed through the large language model.

[0065] When the data volume of the question and answer record is greater than the preset data volume, the question and answer record can be converted into a text vector and stored in the vector library. When executing the current round of question and answer, if the sum of the lengths of the question, primary memory information and secondary memory information of the current round is less than the input window length of the large language model, the text content similar to the question of the current round is obtained from the vector library, and then the question, primary memory information, secondary memory information and text content of the current round are input into the large language model, which can ensure the integrity of the context information.

[0066] Figure 4 is a flowchart of generating text analysis questions based on the self-question-answering form in an embodiment of the present invention, such as Figure 4 As shown, the text analysis question for analyzing the text to be analyzed is generated by the large language model based on the professional knowledge in a self-question-answering format, that is, the above-mentioned step 130, including:

[0067] Step 131 : inputting a second prompt text including at least one second process example into the large language model, where the second process example is an example process for generating an example text analysis question based on example expertise.

[0068] The second process example is a processing process of chain thinking adopted. A second prompt text including at least one second process example is input into the large language model, so that the large language model learns the chain thinking processing process when generating text analysis questions based on professional knowledge, and performs the generation of text analysis text based on the processing process.

[0069] Step 132: Generate an initial analysis question based on the second prompt text and the professional knowledge through the large language model.

[0070] The large language model learns the second process examples in the second prompt text and uses a similar chain thinking process to think about how to raise more comprehensive or higher-order text analysis questions based on existing professional knowledge, first generating initial analysis questions based on professional knowledge.

[0071] In one embodiment of the present invention, the generating of the initial analysis question according to the professional knowledge through the large language model includes: when it is determined based on human-computer interaction that the text analysis question needs to be generated in combination with the text to be analyzed, retrieving the text vector corresponding to the text to be analyzed from a vector library, and obtaining the text content corresponding to the text vector; generating the initial analysis question according to the professional knowledge and the text content through the large language model.

[0072] The text to be analyzed may be pre-embedded to convert the text to be analyzed into a vector representation, obtain a text vector corresponding to the text to be analyzed, and store the text vector in a vector library.

[0073] The question "Do you need to generate the text analysis question in combination with the text to be analyzed?" can be displayed through the human-computer interaction interface, and the user's choice can be received. If the user selects the "yes" option, it is determined that the text analysis question needs to be generated in combination with the text to be analyzed. At this time, the text vector corresponding to the text to be analyzed is retrieved from the vector library, and the text content corresponding to the text vector is obtained. The text content can be the content summarized and refined from the text to be analyzed. The large language model can combine text content and professional knowledge to generate initial analysis questions, and in the subsequent chain thinking processing process, it combines text content, professional knowledge and contextual information of chain thinking processing to process, and finally obtains high-quality text analysis questions.

[0074] Step 133: Obtain a second intermediate answer corresponding to the initial analysis question through the large language model.

[0075] Based on the initial analysis question, the large language model can determine the second intermediate answer based on its own knowledge reserves, or it can obtain the second intermediate answer with the help of search engine tools based on the guidance of the second prompt text.

[0076] When it is necessary to call a search engine to obtain an answer, the large language model will output a search instruction, which includes a search keyword and a search engine call interface. The program calls the search engine call interface to input the search keyword, searches through the search engine, parses and organizes the search results, obtains a second intermediate answer, and returns the second intermediate answer to the large language model.

[0077] Step 134, reflect on the second intermediate answer through the large language model and determine the next second operation.

[0078] The ReAct paradigm is adopted to generate text analysis questions, firstly reflecting on the second intermediate answer through a large language model and determining the second action (i.e., further action to be taken) that needs to be performed next.

[0079] Step 135: Execute the second operation through the large language model to obtain a second operation result, and observe the second operation result.

[0080] Based on the guidance of the second prompt text, the second operation is performed through the large language model to obtain the second operation result. The second operation can be to search for relevant knowledge content with the help of a search engine, or it can be processed with the help of other tools. After obtaining the second operation result, the large language model can observe the effect of the second operation result. After obtaining the second operation result, it can also be provided to the user for viewing through human-computer interaction.

[0081] Step 136, determining whether the text analysis question has been generated based on the observation conclusion, if not, iteratively executing steps 134 to 136 based on the observation conclusion until the text analysis question is generated.

[0082] The large language model can determine whether a high-quality text analysis question has been generated based on the conclusion of the observation, or it can also determine whether a high-quality text analysis question has been generated currently through human-computer interaction. If the conclusion of the sentence observation determines that a high-quality text analysis question has not been generated, based on the conclusion of the observation, iteratively execute steps 134 to 136 to reflect on the conclusion of the observation through the large language model, and determine the second operation of the next step, execute the second operation through the large language model, obtain the second operation result, observe the second operation result, and determine whether the text analysis question has been generated based on the conclusion of the observation, and iteratively execute this process until the text analysis question is generated.

[0083] By inputting the second prompt text into the large language model, the large language model can generate text analysis questions based on professional knowledge based on the prompt of the second process example, using the ReAct paradigm, and can generate high-quality text analysis questions.

[0084] On the basis of the above technical solution, after generating the text analysis question, the method further includes: adding the question and answer record of generating the text analysis question to the second prompt text in the form of the second process example.

[0085] After generating text analysis questions based on professional knowledge, the question and answer records of generating initial analysis questions, obtaining second intermediate answers, reflecting, observing, etc. are used as a second process example, and the second process example is added to the second prompt text to better guide the large language model to generate high-quality text analysis questions based on professional knowledge.

[0086] The method for generating text analysis questions provided by the embodiment of the present invention can be applied to any product or service that analyzes complex long texts. For example, it can be applied to script analysis, understanding and screening. Since scripts are usually long and the script volume is large, the manpower for script evaluation is limited. It is possible to miss excellent scripts that can be accepted by a larger group of people because of someone's subjective views. By using the method for generating text analysis questions provided by the embodiment of the present invention, a more objective text analysis question can be generated, and then the script can be analyzed based on the text analysis question, which can avoid human subjective views and provide more objective results; it can also be applied to plot understanding, by generating text analysis questions for long texts such as plot introductions and lines, and then analyzing long texts such as plot introductions and lines.

[0087] The method for generating text analysis questions provided by the embodiment of the present invention can be applied to Figure 5 In the text analysis framework shown in Figure 5 As shown, the following processes are performed in the text analysis framework: large language models (LLMs) role setting (set the role); prompt rendering (Prompt rendering), using large language models (LLMs) to raise text analysis questions; using large language models (LLMs) to answer text analysis questions and obtain text analysis answers (Documents retrieval); text analysis answer selection and evaluation (Answer choice / evaluation); based on the multi-path feedback that meets the expectations in the text analysis question raising stage, the text analysis question answering stage, the text analysis answer selection and evaluation stage, multi-layer sample enhancement is performed in the text analysis question raising stage, the text analysis question answering stage, the text analysis answer selection and evaluation stage. The method for generating text analysis questions in the embodiment of the present invention can be applied to Figure 5 The stage of using large language models to raise text analysis problems.

[0088] (1) Role Setting of Large Language Model

[0089] Set the role of the large language model according to the type of text to be analyzed. Let the large language model know the set role through prompt rendering. For example, using OpenAI, the role of the large language model can be set through the initial system role.

[0090] For example, in engineering implementation, two class instances can be created, each of which uses the call of the large language model as a method that can be used by itself. Each instance has its own Prompt attribute, and the role of the large language model is set as the questioner and the answerer respectively.

[0091] (2) Raising text analysis questions

[0092] For the questioner, the Prompt paradigm is combined with the set identity role to enable the large language model to independently raise text analysis questions corresponding to the text to be analyzed. The guidance of the Prompt paradigm allows the large language model to think about which text analysis questions should be raised under the current role. Among them, the process of allowing the large language model to raise text analysis questions through the guidance of the Prompt paradigm can be implemented using the method for generating text analysis questions provided in the embodiment of the present invention.

[0093] Optionally, the Prompt paradigm can be a chain thinking method such as COT (Chain of Thoughts); or it can be a more complex SA (Self-Ask) + SE (Search Engine) method, which also shows the thinking process of why this role asks a certain text analysis question, and can also use the search engine to assist in finding professional questions that can be used for text analysis in the professional field related to the role during the thinking process; it can also further use the ReAct (Reasoning and Action) method to further decompose specific text analysis problems, and ask single-hop or even recursive questions to the decomposed questions to enhance the large language model's understanding of text analysis problems; of course, other methods can also be used to analyze the text to be analyzed in order to raise text analysis questions.

[0094] (3) Answer text analysis questions

[0095] As an answerer, you can build a tool library that can be used by large language models, and use your own knowledge, network resources, and text vector libraries to answer text analysis questions based on the question and answer context.

[0096] Define the action space (Action Space) that can be called by the respondent. In addition to the API (Application Program Interface) that can be called by the third party (such as the search engine interface, etc.), the reasoning interface of various models can also be registered in the Action Space.

[0097] Guided by the prompt paradigm, the large language model can be used to understand text analysis questions based on the questioner and answerer's question and answer history, and choose the way to answer text analysis questions. Answers can be made based on the actual knowledge accumulated during pre-training; or based on the understanding of the interface description in the Action Space, the interface call can be used to solve the problem. For example, for understanding the plot, relevant information can be obtained with the help of search engines. For the analysis of complex and long texts, question matching based on the text vector library and the integration and answer of related content can be used to get rid of the dilemma that the large language model cannot quickly and cost-effectively inject a large amount of vertical information and the context window limitation during application.

[0098] By analyzing complex long texts, we can obtain the refined content of the text and build a text vector library based on the refined content of the text. The subsequent large language model can convert text analysis questions into text vector library requests, thereby obtaining answers based on the text vector library.

[0099] (4) Text Analysis Answer Evaluation

[0100] Given that the analysis of complex and long texts is not a simple logical or mathematical problem, it usually does not have a "standard answer" in the traditional sense, and it focuses more on the analysis process rather than just the final result. Therefore, an evaluation mechanism can be established for the answers to the above questions to obtain the scores of the text analysis answers.

[0101] (5) Multi-layer sample enhancement

[0102] If the score of the text analysis answer is in line with expectations, the input and output in (2) to (4) above are added as feedback samples to the corresponding links to achieve sample reflux, so that the large language model can achieve better performance.

[0103] Figure 6 is a schematic diagram of the structure of a device for generating text analysis questions provided by an embodiment of the present invention, such as Figure 6 As shown, the device for generating the text analysis problem includes:

[0104] A role acquisition module 510, for acquiring an identity role of a large language model, wherein the identity role corresponds to a text type of a text to be analyzed;

[0105] A professional knowledge acquisition module 520 is used to acquire professional knowledge corresponding to the text type based on the identity role through the large language model in a self-question-answering format;

[0106] The analysis question generation module 530 is used to generate text analysis questions for analyzing the text to be analyzed based on the professional knowledge and in a self-question-answering format through the large language model, and the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

[0107] Optionally, the professional knowledge acquisition module includes:

[0108] A first prompt input unit, configured to input a first prompt text including at least one first process example into the large language model, wherein the first process example is an example process for acquiring corresponding example expertise based on an example role;

[0109] A professional question generating unit, configured to generate professional knowledge questions corresponding to the identity role through the large language model according to the first prompt text;

[0110] A first intermediate answer obtaining unit, configured to obtain a first intermediate answer corresponding to the professional knowledge question through the large language model;

[0111] A first reflection unit, configured to reflect on the first intermediate answer through the large language model and determine a first operation for the next step;

[0112] a first action observation unit, configured to perform the first operation through the large language model, obtain a first operation result, and observe the first operation result;

[0113] The first iterative control unit is used to iteratively execute the steps of reflecting on the conclusion of the observation, performing the first operation and observing, when it is determined according to the conclusion of the observation that the professional knowledge corresponding to the text type has not been acquired, based on the conclusion of the observation, until the professional knowledge corresponding to the text type is acquired.

[0114] Optionally, the first intermediate answer obtaining unit is specifically used to:

[0115] Inputting the professional knowledge question into the large language model, and receiving a search instruction output by the large language model;

[0116] According to the search keywords in the search instruction, a search is performed through a search engine, and the search results are parsed to obtain the first intermediate answer.

[0117] Optionally, the device further comprises:

[0118] The first prompt updating module is used to add the question and answer record of the acquired professional knowledge into the first prompt text in the form of the first process example.

[0119] Optionally, the professional knowledge acquisition module includes:

[0120] a memory information generating unit, configured to use, before executing the current round of question and answer, the question and answer records of a preset length before the current round as primary memory information through the large language model, and generate summary information corresponding to the question and answer records before the preset length, and use the summary information as secondary memory information;

[0121] The question-answering execution unit is used to input the question of the current round, the primary memory information and the secondary memory information into the large language model, and execute the question-answering of the current round through the large language model.

[0122] Optionally, the device further comprises:

[0123] A question and answer record vectorization module, used for converting the question and answer record before the current round into a text vector if the data amount of the question and answer record before the current round is greater than a preset data amount, and storing the text vector in a vector library;

[0124] The question-answer execution unit is specifically used for:

[0125] When the sum of the lengths of the question of the current round, the primary memory information, and the secondary memory information is less than the input window length of the large language model, converting the question of the current round into a question vector, and retrieving a target text vector having a similarity with the question vector greater than a similarity threshold from the vector library, and acquiring text content corresponding to the target text vector;

[0126] The question of the current round, the primary memory information, the secondary memory information and the text content are input into the large language model, and the question and answer of the current round is performed through the large language model.

[0127] Optionally, the professional knowledge acquisition module includes:

[0128] A second prompt input unit, configured to input a second prompt text including at least one second process example into the large language model, wherein the second process example is an example process for generating an example text analysis question based on example expertise;

[0129] an initial analysis question generating unit, configured to generate an initial analysis question according to the second prompt text and the professional knowledge through the large language model;

[0130] A second intermediate answer obtaining unit, configured to obtain a second intermediate answer corresponding to the initial analysis question through the large language model;

[0131] A second reflection unit, used to reflect on the second intermediate answer through the large language model and determine a second operation for the next step;

[0132] a second action observation unit, configured to perform the second operation through the large language model, obtain a second operation result, and observe the second operation result;

[0133] The second iterative control unit is used to iteratively execute the above-mentioned reflection, execution of the second operation and observation steps based on the conclusion of the observation when it is determined that the text analysis question has not been generated according to the conclusion of the observation, until the text analysis question is generated.

[0134] Optionally, the initial analysis question generating unit is specifically used for:

[0135] When it is determined based on human-computer interaction that the text analysis problem needs to be generated in combination with the text to be analyzed, a text vector corresponding to the text to be analyzed is retrieved from a vector library, and text content corresponding to the text vector is obtained;

[0136] The initial analysis question is generated by the large language model according to the professional knowledge and the text content.

[0137] Optionally, the device further comprises:

[0138] The second prompt updating module is used to add the question and answer record of generating the text analysis question into the second prompt text in the form of the second process example.

[0139] The device for generating text analysis questions provided by the present embodiment obtains the identity role of a large language model, and based on the identity role, obtains the professional knowledge that the identity role should have through the large language model in the form of self-questioning and answering, and generates a text analysis text for analyzing a text to be analyzed through the large language model in the form of self-questioning and answering according to the professional knowledge. Since the professional knowledge that the identity role should have can be obtained based on the identity role through the large language model, high-quality text analysis questions can be generated based on the professional knowledge, and no fine-tuning training is required, which can reduce the time for generating text analysis questions, improve the efficiency of generating text analysis questions, and do not require a large amount of manpower, which can reduce costs.

[0140] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0141] Memory 603, used for storing computer programs;

[0142] The processor 601 is used to execute the program stored in the memory 603 to implement the following steps:

[0143] Obtaining an identity role of a large language model, wherein the identity role corresponds to a text type of a text to be analyzed;

[0144] According to the identity role, the professional knowledge corresponding to the text type is acquired through the large language model in a self-question-answering format;

[0145] According to the professional knowledge, text analysis questions for analyzing the text to be analyzed are generated by the large language model in a self-question-answering format, and the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

[0146] Optionally, the acquiring, according to the identity role, the professional knowledge corresponding to the text type through the large language model in a self-question-answering format includes:

[0147] Inputting a first prompt text including at least one first process example into the large language model, wherein the first process example is an example process for acquiring corresponding example expertise based on an example role;

[0148] Generate professional knowledge questions corresponding to the identity role through the large language model according to the first prompt text;

[0149] Acquire a first intermediate answer corresponding to the professional knowledge question through the large language model;

[0150] Reflecting on the first intermediate answer through the large language model and determining a first operation for the next step;

[0151] Performing the first operation through the large language model to obtain a first operation result, and observing the first operation result;

[0152] When it is determined based on the observation conclusion that the professional knowledge corresponding to the text type has not been acquired, based on the observation conclusion, iteratively execute the steps of reflecting on the observation conclusion, executing the first operation and observing until the professional knowledge corresponding to the text type is acquired.

[0153] Optionally, the obtaining, by using the large language model, a first intermediate answer corresponding to the professional knowledge question includes:

[0154] Inputting the professional knowledge question into the large language model, and receiving a search instruction output by the large language model;

[0155] According to the search keywords in the search instruction, a search is performed through a search engine, and the search results are parsed to obtain the first intermediate answer.

[0156] Optionally, after acquiring the professional knowledge corresponding to the text type, the method further includes:

[0157] The question and answer record of the acquired professional knowledge is added to the first prompt text in the form of the first process example.

[0158] Optionally, the acquiring, according to the identity role, the professional knowledge corresponding to the text type through the large language model in a self-question-answering format includes:

[0159] Before executing the current round of question and answer, the large language model uses the question and answer records of a preset length before the current round as primary memory information, generates summary information corresponding to the question and answer records before the preset length, and uses the summary information as secondary memory information;

[0160] The question of the current round, the primary memory information and the secondary memory information are input into the large language model, and the question and answer of the current round is performed through the large language model.

[0161] Optionally, also include:

[0162] If the data volume of the question-and-answer record before the current round is greater than the preset data volume, converting the question-and-answer record before the current round into a text vector, and storing the text vector in a vector library;

[0163] The step of inputting the current round of questions, the primary memory information, and the secondary memory information into the large language model, and performing the current round of question and answering through the large language model includes:

[0164] When the sum of the lengths of the question of the current round, the primary memory information, and the secondary memory information is less than the input window length of the large language model, converting the question of the current round into a question vector, and retrieving a target text vector having a similarity with the question vector greater than a similarity threshold from the vector library, and acquiring text content corresponding to the target text vector;

[0165] The question of the current round, the primary memory information, the secondary memory information and the text content are input into the large language model, and the question and answer of the current round is performed through the large language model.

[0166] Optionally, the step of generating, by the large language model according to the professional knowledge and in a self-question-and-answer format, a text analysis question for analyzing the text to be analyzed includes:

[0167] inputting a second prompt text including at least one second process example into the large language model, the second process example being an example process for generating an example text analysis question based on example expertise;

[0168] According to the second prompt text, generating an initial analysis question by using the large language model according to the professional knowledge;

[0169] Acquire a second intermediate answer corresponding to the initial analysis question through the large language model;

[0170] Reflecting on the second intermediate answer through the large language model and determining a second operation for the next step;

[0171] Performing the second operation through the large language model to obtain a second operation result, and observing the second operation result;

[0172] When it is determined according to the conclusion of the observation that the text analysis question has not been generated, based on the conclusion of the observation, the above-mentioned steps of reflecting, performing the second operation and observing are iteratively performed until the text analysis question is generated.

[0173] Optionally, generating initial analysis questions according to the professional knowledge through the large language model includes:

[0174] When it is determined based on human-computer interaction that the text analysis problem needs to be generated in combination with the text to be analyzed, a text vector corresponding to the text to be analyzed is retrieved from a vector library, and text content corresponding to the text vector is obtained;

[0175] The initial analysis question is generated by the large language model according to the professional knowledge and the text content.

[0176] Optionally, after generating the text analysis question, the method further includes:

[0177] The question and answer record for generating the text analysis question is added to the second prompt text in the form of the second process example.

[0178] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0179] The communication interface is used for communication between the above electronic device and other devices.

[0180] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0181] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0182] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method for generating text analysis problems described in any of the above embodiments.

[0183] In another embodiment provided by the present invention, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the method for generating text analysis questions described in any one of the above embodiments.

[0184] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0185] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0186] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0187] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for generating text analysis questions, characterized in that: include: Obtaining an identity role of a large language model, wherein the identity role corresponds to a text type of a text to be analyzed; According to the identity role, the professional knowledge corresponding to the text type is acquired through the large language model in a self-question-answering format; According to the professional knowledge, text analysis questions for analyzing the text to be analyzed are generated by the large language model in a self-question-answering format, and the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

2. The method according to claim 1, characterized in that The acquiring, according to the identity role, the professional knowledge corresponding to the text type through the large language model in a self-question-answering form includes: Inputting a first prompt text including at least one first process example into the large language model, wherein the first process example is an example process for acquiring corresponding example expertise based on an example role; According to the first prompt text, generating professional knowledge questions corresponding to the identity role through the large language model; Acquire a first intermediate answer corresponding to the professional knowledge question through the large language model; Reflecting on the first intermediate answer through the large language model and determining a first operation for the next step; Performing the first operation through the large language model to obtain a first operation result, and observing the first operation result; When it is determined based on the observation conclusion that the professional knowledge corresponding to the text type has not been acquired, based on the observation conclusion, iteratively execute the steps of reflecting on the observation conclusion, executing the first operation and observing until the professional knowledge corresponding to the text type is acquired.

3. The method according to claim 2, characterized in that The obtaining, by using the large language model, a first intermediate answer corresponding to the professional knowledge question includes: Inputting the professional knowledge question into the large language model, and receiving a search instruction output by the large language model; According to the search keywords in the search instruction, a search is performed through a search engine, and the search results are parsed to obtain the first intermediate answer.

4. The method according to claim 2, characterized in that: After acquiring the professional knowledge corresponding to the text type, the method further includes: The question and answer record of the acquired professional knowledge is added to the first prompt text in the form of the first process example.

5. The method according to claim 1, characterized in that The acquiring, according to the identity role, the professional knowledge corresponding to the text type through the large language model in a self-question-answering form includes: Before executing the current round of question and answer, the large language model uses the question and answer records of a preset length before the current round as primary memory information, generates summary information corresponding to the question and answer records before the preset length, and uses the summary information as secondary memory information; The current round of questions, the primary memory information and the secondary memory information are input into the large language model, and the current round of questions and answers is performed through the large language model.

6. The method according to claim 5, characterized in that Also includes: If the data volume of the question-and-answer record before the current round is greater than the preset data volume, converting the question-and-answer record before the current round into a text vector, and storing the text vector in a vector library; The step of inputting the current round of questions, the primary memory information, and the secondary memory information into the large language model, and performing the current round of question and answering through the large language model includes: When the sum of the lengths of the question of the current round, the primary memory information, and the secondary memory information is less than the input window length of the large language model, converting the question of the current round into a question vector, and retrieving a target text vector having a similarity with the question vector greater than a similarity threshold from the vector library, and acquiring text content corresponding to the target text vector; The question of the current round, the primary memory information, the secondary memory information and the text content are input into the large language model, and the question and answer of the current round is performed through the large language model.

7. The method according to claim 1, characterized in that The step of generating a text analysis question for analyzing the text to be analyzed based on the professional knowledge and in a self-question-answering format using the large language model includes: inputting a second prompt text including at least one second process example into the large language model, the second process example being an example process for generating an example text analysis question based on example expertise; According to the second prompt text, generating an initial analysis question by using the large language model according to the professional knowledge; Acquire a second intermediate answer corresponding to the initial analysis question through the large language model; Reflecting on the second intermediate answer through the large language model and determining a second operation for the next step; Performing the second operation through the large language model to obtain a second operation result, and observing the second operation result; When it is determined according to the conclusion of the observation that the text analysis question has not been generated, based on the conclusion of the observation, the above-mentioned steps of reflecting, performing the second operation and observing are iteratively performed until the text analysis question is generated.

8. The method according to claim 7, characterized in that The generating of initial analysis questions based on the professional knowledge by using the large language model includes: When it is determined based on human-computer interaction that the text analysis problem needs to be generated in combination with the text to be analyzed, a text vector corresponding to the text to be analyzed is retrieved from a vector library, and text content corresponding to the text vector is obtained; The initial analysis question is generated by the large language model according to the professional knowledge and the text content.

9. The method according to claim 7, characterized in that: After generating the text analysis question, the method further includes: The question and answer record for generating the text analysis question is added to the second prompt text in the form of the second process example.

10. A device for generating text analysis questions, characterized in that: include: A role acquisition module, used to acquire the identity role of the large language model, wherein the identity role corresponds to the text type of the text to be analyzed; A professional knowledge acquisition module, used to acquire professional knowledge corresponding to the text type based on the identity role through the large language model in a self-question-answering format; An analysis question generation module is used to generate text analysis questions for analyzing the text to be analyzed based on the professional knowledge and in a self-question-answering format through the large language model, wherein the text analysis questions are used to ask questions to the large language model to analyze the text to be analyzed.

11. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 9 when executing a program stored in a memory.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising instructions, characterized in that: When the computer program product is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Question and answer processing method, system and equipment and storage medium

    CN117473057A

  • Question and answer pair construction method and device, question and answer method and device, electronic equipment and storage medium

    CN117573830A

Cited By

  • Device for controlling results of model questions and answers through role definition

    CN120804273A