Prompt text expansion method and apparatus, electronic device, and storage medium

By automatically extracting entities and keywords, generating extended prompt text using a pre-trained model, and determining whether to continue expanding based on the similarity between the answer text and the standard answer, this technology solves the problems of low efficiency and high cost in generating prompt text in existing technologies, and achieves more efficient and accurate text generation.

CN116738250BActive Publication Date: 2026-01-23GUANGZHOU HUYA TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310716018.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-15
Publication Date
2026-01-23
Estimated Expiration
2043-06-15

AI Technical Summary

Technical Problem

Existing methods for generating prompt text by manually writing or using fixed templates are inefficient, costly, and poorly adaptable, making it difficult to adapt to different user needs and changes in the field, resulting in inaccurate generated text.

Method used

By extracting entities and/or keywords from the initial prompt text, expanding it using a pre-trained model to generate expanded prompt text, and determining whether to continue expanding based on the similarity between the pre-trained model's answer text and the standard answer, until a preset condition is met.

Benefits of technology

It improves the efficiency of generating prompt text, reduces costs, and can better adapt to user needs and changes in the field, generating more comprehensive and accurate text.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116738250B_ABST
    Figure CN116738250B_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a prompt text expansion method and device, electronic equipment and storage medium, relating to the technical field of natural language processing. The method automatically extracts entities and / or keywords in the initial prompt text, automatically generates an expanded prompt text based on the entities and / or keywords through a pre-trained model, and determines whether to continue the prompt text expansion according to the similarity between the answer text generated by the pre-trained model in response to the target question in the expanded prompt text and the standard answer, so that the final expanded prompt text is more comprehensive and accurate, more fully describes the user's needs and intentions, and further guides the model to generate more accurate and valuable text. In this way, the generation efficiency of the prompt text is improved, the cost is reduced, and different user needs and field changes can be adapted.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, in particular to a prompt text expansion method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of natural language processing technology, people's demand for generating natural language text is also increasing. The emergence of Prompt is to meet this demand, which can be used as a keyword or phrase to guide the model to generate text. With the continuous progress of artificial intelligence technology, the application scenarios of Prompt are also expanding. In addition to text generation, it can also be applied in question and answer systems, machine translation, intelligent customer service and other fields. The development of Prompt not only promotes the progress of natural language processing technology, but also provides more possibilities for the application of artificial intelligence technology in various fields.

[0003] In the prior art, Prompt is mainly manually written or uses a fixed template or format as Prompt. Manual writing is manually written by professionals, which requires professionals to invest a lot of manpower and time, and requires professional domain knowledge and language ability; in addition, the writing of Prompt also needs to be constantly updated and optimized to adapt to user needs and changes in the field, which is not only inefficient, but also costly, difficult to expand and popularize. Using a fixed template or format makes it difficult to adapt to different user needs and changes in the field, resulting in inaccurate generated text that does not match user needs and intentions. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a prompt text expansion method, device, electronic equipment and storage medium to solve the problems of low efficiency, high cost and poor adaptability in the prior art of manually writing or using templates or formats to generate prompt text.

[0005] To achieve the above purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0006] In a first aspect, the present application provides a prompt text expansion method, which comprises:

[0007] extracting entities and / or keywords in the initial prompt text;

[0008] expanding the entities and / or keywords through a pre-trained model to obtain an expanded prompt text;

[0009] obtaining an answer text generated by the pre-trained model according to the expanded prompt text; the answer text is an answer obtained by the pre-trained model to a target question in the expanded prompt text;

[0010] According to the answer text, the standard answer corresponding to the target question, the extended prompt text, the entity and / or keyword, and the pre-trained model, a new extended prompt text is obtained until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition.

[0011] In an optional implementation, the extending the entity and / or keyword by the pre-trained model to obtain the extended prompt text comprises:

[0012] generating a first prompt text according to a first preset prompt template and the entity and / or keyword;

[0013] inputting the first prompt text into the pre-trained model, guiding the pre-trained model to extend the entity and / or keyword through the first prompt text, and outputting an extended prompt text.

[0014] In an optional implementation, the obtaining the answer text generated by the pre-trained model according to the extended prompt text comprises:

[0015] determining a number of standard answers corresponding to a target question in the extended prompt text;

[0016] inputting the extended prompt text into the pre-trained model multiple times according to the number of standard answers, guiding the pre-trained model to answer the target question in the extended prompt text through the extended prompt text, and obtaining an answer text output by the pre-trained model each time; wherein the number of answer texts is the same as the number of standard answers.

[0017] In an optional implementation, the obtaining a new extended prompt text according to the answer text, the standard answer corresponding to the target question, the extended prompt text, the entity and / or keyword, and the pre-trained model until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition comprises:

[0018] After obtaining the answer text output by the pre-trained model each time, a similarity score between the answer text and each standard answer corresponding to the target question is calculated, and a highest similarity score is determined from each similarity score corresponding to the answer text; wherein each answer text corresponds to a highest similarity score;

[0019] a lowest target similarity score is selected from the highest similarity scores corresponding to each answer text according to a preset ratio; the number of selected target similarity scores is determined according to the preset ratio and the number of answer texts;

[0020] If at least one of the target similarity scores is less than a preset threshold, a target answer text corresponding to the target similarity score less than the preset threshold and a target standard answer are obtained, a new extended prompt text is generated based on the target answer text, the target standard answer, the entity and / or keyword, and the extended prompt text by the pre-training model until all target similarity scores selected according to the answer text generated by the pre-training model are greater than or equal to the preset threshold.

[0021] In an optional implementation, the generating, by the pre-training model, of the new extended prompt text based on the target answer text, the target standard answer, the entity and / or keyword, and the extended prompt text comprises:

[0022] generating sample data according to the target answer text, the target standard answer, and the target question, and generating a second prompt text according to the sample data, the entity and / or keyword, the extended prompt text, and a second preset prompt template;

[0023] inputting the second prompt text into the pre-training model, and guiding the pre-training model to adjust the extended prompt text based on the entity and / or keyword and the sample data and output a new extended prompt text through the second prompt text.

[0024] In an optional implementation, the determining of the number of standard answers corresponding to the target question in the extended prompt text comprises:

[0025] finding a target preset question matching the target question in the extended prompt text in a preset database, taking all standard answers corresponding to the target preset question as the standard answers corresponding to the target question, and obtaining the number of standard answers corresponding to the target question; the preset database stores a plurality of preset questions and a standard answer corresponding to each of the preset questions.

[0026] In a second aspect, the present application provides a prompt text extension device, which comprises:

[0027] an extraction module configured to extract an entity and / or a keyword in an initial prompt text;

[0028] an extension module configured to extend the entity and / or the keyword by a pre-training model to obtain an extended prompt text;

[0029] an answer text acquisition module configured to acquire an answer text generated by the pre-training model based on the extended prompt text; the answer text is an answer obtained by the pre-training model in response to a target question in the extended prompt text;

[0030] The prompt text adjustment module is configured to obtain a new extended prompt text according to the answer text, the standard answer corresponding to the target question, the extended prompt text, the entity and / or keyword, and the pre-trained model until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition.

[0031] In an optional implementation, the answer text acquisition module is configured to determine the number of standard answers corresponding to the target question in the extended prompt text; input the extended prompt text into the pre-trained model multiple times according to the number of standard answers, guide the pre-trained model to answer the target question in the extended prompt text through the extended prompt text, and acquire the answer text output by the pre-trained model each time; wherein the number of answer texts is the same as the number of standard answers.

[0032] The prompt text adjustment module is configured to, after acquiring the answer text output by the pre-trained model each time, calculate the similarity scores between the answer text and each standard answer corresponding to the target question, and determine the highest similarity score from the similarity scores corresponding to the answer text; wherein each answer text corresponds to a highest similarity score; select the lowest target similarity score from the highest similarity scores corresponding to each answer text according to a preset proportion; the number of selected target similarity scores is determined according to the preset proportion and the number of answer texts; if at least one target similarity score is less than a preset threshold, acquire the target answer text and the target standard answer corresponding to the target similarity score less than the preset threshold, generate a new extended prompt text based on the target answer text, the target standard answer, the entity and / or keyword, and the extended prompt text through the pre-trained model, until all target similarity scores selected according to the answer text generated by the pre-trained model are greater than or equal to the preset threshold.

[0033] In a third aspect, the present application provides an electronic device, comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to implement the steps of the prompt text extension method according to any one of the preceding embodiments.

[0034] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the prompt text extension method according to any one of the preceding embodiments.

[0035] The prompt text expansion method, device, electronic equipment and storage medium provided by the embodiments of the present application, the method comprises extracting entities and / or keywords in the initial prompt text, expanding the entities and / or keywords through a pre-trained model to obtain an expanded prompt text, and obtaining an answer text generated by the pre-trained model according to the expanded prompt text; the answer text is an answer obtained by the pre-trained model in response to a target question in the expanded prompt text; according to the answer text, a standard answer corresponding to the target question, the expanded prompt text, the entities and / or keywords and the pre-trained model, a new expanded prompt text is obtained until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition. By automatically extracting entities and / or keywords, generating an expanded prompt text based on a pre-trained model, and determining whether to continue expanding the prompt text according to the similarity between the answer text generated by the pre-trained model and the standard answer, the expanded prompt text obtained finally is more comprehensive and accurate, and more fully describes the needs and intentions of the user, thereby guiding the model to generate more accurate and valuable text. In this way, the generation efficiency of the prompt text is improved, the cost is reduced, and different user needs and field changes can be adapted.

[0036] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0038] Figure 1 A flowchart of a prompt text expansion method provided by an embodiment of the present application is shown;

[0039] Figure 2 Another flowchart of a prompt text expansion method provided by an embodiment of the present application is shown;

[0040] Figure 3 Another flowchart of a prompt text expansion method provided by an embodiment of the present application is shown;

[0041] Figure 4 Another flowchart of a prompt text expansion method provided by an embodiment of the present application is shown;

[0042] Figure 5 A functional module diagram of a prompt text expansion device provided by an embodiment of the present application is shown;

[0043] Figure 6 A block diagram of an electronic device is shown.

[0044] Icon: 100-electronic device; 110-memory; 120-processor; 130-communication module; 600-prompt text expansion apparatus; 610-extraction module; 620-expansion module; 630-answer text acquisition module; 640-prompt text adjustment module. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0046] Therefore, the detailed description of the embodiments of the present application provided in the accompanying drawings below is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0047] It should be noted that the relational terms such as first and second and the like are used only to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations exist in any such actual relationship or order. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a series of elements includes not only those elements, but also other elements not explicitly listed, or other elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus including the element.

[0048] Large Language Model (LLM) is a natural language processing model based on deep learning technology, which can automatically generate fluent and accurate natural language text. These models are based on deep neural networks, which learn a large amount of language data to predict the probability of the next word or character, thereby generating natural language text. Among them, the most famous large language model is the GPT series model developed by OpenAI, including GPT-1, GPT-2, GPT-3 and other versions. These models use the Transformer architecture, using a large amount of pre-training data and self-supervised learning techniques, and have achieved excellent performance in a variety of natural language processing tasks.

[0049] With the development of natural language processing technology, the demand for generating natural language text is also increasing. The emergence of Prompt is to meet this demand, which can be used as a keyword or phrase to guide the model to generate text. The range of traditional Prompt is small, usually only including one or several keywords or phrases, which is difficult to fully describe the user's needs and intentions. If the Prompt is not comprehensive or accurate enough, it may lead to the generated text not matching the user's needs and intentions, wasting the user's time and effort.

[0050] In the prior art, the Prompt is mainly manually written or uses a fixed template or format as the Prompt. Manually writing requires professionals to have professional domain knowledge and language skills, and requires a lot of manpower and time, and also needs to be constantly updated and optimized to adapt to user needs and domain changes, so it is inefficient and costly, and difficult to expand and popularize. Using a fixed template or format makes it difficult to adapt to different user needs and domain changes, resulting in inaccurate or less useful generated text that does not match the user's needs and intentions.

[0051] Therefore, the embodiments of the present application provide a prompt text expansion method and device, electronic equipment and storage medium, which can automatically extract entities and / or keywords in the initial prompt text, automatically generate expanded prompt text based on the entities and / or keywords through a pre-trained model, and determine whether to continue the prompt text expansion according to the similarity between the answer text generated by the pre-trained model answering the target question in the expanded prompt text and the standard answer, so that the final expanded prompt text is more comprehensive and accurate, and more fully describes the user's needs and intentions, thereby guiding the model to generate more accurate and valuable text. In this way, the generation efficiency of the prompt text is improved, the cost is reduced, and it can also adapt to different user needs and domain changes.

[0052] In the following, the embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0053] Please refer toFigure 1 A flowchart of a prompt text expansion method provided by an embodiment of the present application is shown. It should be noted that the prompt text expansion method of the embodiment of the present application is not limited to the specific order shown and described below, and it should be understood that in other embodiments, the order of some steps of the prompt text expansion method of the embodiment of the present application can be exchanged according to actual needs, or some steps can be omitted or deleted. The prompt text expansion method can be applied in electronic devices such as personal computers (PCs), tablet computers, smart phones, etc. The specific process shown below will be described in detail. Figure 1 Figure 1

[0054] Step S101, extracting entities and / or keywords in the initial prompt text.

[0055] In this embodiment, the initial prompt text can be input by the user or automatically generated by the electronic device based on events occurring in the current application scenario. Taking the digital human game commentary scenario as an example, the electronic device can automatically generate a simple Prompt_init (i.e., initial prompt text) according to a preset template or format based on the identified game results (game win, game loss), performance (e.g., 0 kills, 9 deaths, 2 assists), etc. by intercepting the screen image and performing recognition. For example, the initial prompt text can include the following content: "The game result is a win, the anchor lies and wins, and the performance is very poor. Please comment on the anchor's performance in a sarcastic style." It can be understood that the electronic device can automatically generate different initial prompt texts according to different events.

[0056] In actual applications, due to the ambiguity and polysemy of natural language, a simple Prompt often cannot fully describe the user's needs and intentions. For example, the simple Prompt "buy a car" may contain many aspects of information, such as car model, price, brand, color, etc. If only this simple Prompt is used, it may not accurately meet the user's needs. Therefore, in the embodiment of the present application, entities and / or keywords in the initial prompt text need to be extracted for expansion in the next step to obtain more rich content.

[0057] In one embodiment, natural language processing technology can be used to extract entities and keywords. With the help of natural language processing technology, relevant information such as part of speech and named entities can be extracted from the input simple Prompt, thereby realizing the recognition of entities and keywords. The advantage of this is that by automatically extracting entities and keywords, the user's needs and intentions can be more accurately understood, thereby better generating text that meets the user's needs, improving the accuracy and quality of text generation, and improving the user experience. ​​

[0058] Among them, common entity recognition technologies include named entity recognition and entity recognition. Named entity recognition can identify entities with fixed names or specific semantics in text, such as names, place names, organization names, etc. Entity recognition can identify all entities in the text, such as dates, numbers, currency symbols, etc. Keywords can be identified through text classification, text clustering, etc.

[0059] In one example, for the extraction of entities and keywords, a first general template can be pre-set, and the electronic device generates a prompt text Prompt1 for guiding the model to extract entities and keywords according to the first general template and the content in the initial prompt text. For example, Prompt1 can include "Please extract entities and keywords according to the following text content". By calling the API interface corresponding to the pre-training model (a pre-trained large language model), Prompt1 is input into the pre-training model, and the pre-training model can complete the extraction of entities and / or keywords in the initial prompt text under the guidance of Prompt1. It should be noted that the first general template is a relatively fine template pre-set for text entity and keyword extraction in various scenarios, and the Prompt1 generated in actual application can include more content, not limited to the above example.

[0060] In another embodiment, in addition to automatically extracting entities and keywords, manual annotation or a combination of manual and automatic annotation methods can also be used to obtain entity and keyword information, that is, the specific implementation of the embodiment of the present application is not limited to the extraction of entities and keywords.

[0061] It should be noted that in actual application, for the case where no entity or keyword is extracted, the step of obtaining Prompt_init can be directly returned, and the subsequent process is executed after a new Prompt_init is obtained; or other technologies can be used to analyze Prompt_init. For example, text classification technology can be used to classify Prompt_init, and then identify the purpose and intention of Prompt_init.

[0062] Step S102, expanding the entity and / or keyword through the pre-training model to obtain an expanded prompt text.

[0063] In this embodiment, after the electronic device extracts the entity and / or keyword in the initial prompt text, the entity and / or keyword is expanded through the API interface corresponding to the pre-training model to obtain an expanded prompt text. The expanded prompt text will contain more entities and keywords, thereby more comprehensively describing the user's requirements and intentions.

[0064] It can be seen that the initial prompt text is automatically expanded into more comprehensive and accurate content using the pre-trained model, which realizes automatic expansion of Prompt, effectively avoids manual intervention, and improves production efficiency. At the same time, since the expansion of Prompt is automatically completed based on the pre-trained model, the expansion result is more objective and accurate. If automatic expansion is not performed, the generated text may be far from the actual needs and intentions of the user, resulting in user dissatisfaction. By automatically expanding Prompt, the user's needs and intentions can be better captured, and more accurate and valuable text can be generated.

[0065] In step S103, the answer text generated by the pre-trained model according to the expanded prompt text is obtained; the answer text is the answer obtained by the pre-trained model to the target question in the expanded prompt text.

[0066] In this embodiment, considering that the pre-trained model may not be able to obtain an expanded prompt text that meets the user's needs by performing Prompt expansion only once, the generated expanded prompt text needs to be input into the pre-trained model, the target question in the expanded prompt text is answered by the pre-trained model, and the corresponding answer text is obtained. According to whether the answer text output by the pre-trained model belongs to the answer that the user wants, it is further judged whether it is necessary to continue to perform Prompt expansion.

[0067] Among them, the target question in the expanded prompt text can be understood as the question described in the initial prompt text. For example, the initial prompt text includes the content "the game result is victory, the anchor wins, the record is very poor, please explain the anchor's performance in a satirical style", and the expanded prompt text is obtained based on the entity and / or keyword in the initial prompt text. The target question in the expanded prompt text can be "satirize the anchor's poor performance and return the corresponding explanation content".

[0068] In step S104, according to the answer text, the standard answer corresponding to the target question, the expanded prompt text, the entity and / or keyword, and the pre-trained model, a new expanded prompt text is obtained until the similarity between the answer text generated by the pre-trained model and the standard answer reaches the preset condition.

[0069] In the embodiment, when the similarity between the answer text generated by the pre-training model and the standard answer does not meet the preset condition, it indicates that the difference between the answer text generated by the pre-training model and the standard answer is relatively large, and the current extended prompt text cannot meet the user demand, and the pre-training model needs to continue to perform Prompt expansion. The pre-training model performs Prompt expansion again according to the answer text, the standard answer corresponding to the target question, and the current extended prompt text, the extracted entity and / or keyword, adjusts the current extended prompt text, and outputs a new extended prompt text. The electronic device inputs the new extended prompt text into the pre-training model, obtains the answer text output by the pre-training model again, and judges whether the similarity between the answer text and the standard answer meets the preset condition; the above steps are repeated, so that the answer text output by the pre-training model based on the extended prompt text gradually approaches the standard answer until the preset condition is met, and the final extended prompt text is obtained.

[0070] After obtaining the final extended prompt text, the pre-training model is started to generate text using the extended prompt text as input, and the generated text will be more accurate and valuable, meeting the user's demand and intention. If automatic expansion is not performed, the generated text may be far from the user's actual demand and intention, resulting in user dissatisfaction.

[0071] It can be seen that the prompt text expansion method provided in the embodiment includes extracting an entity and / or a keyword in an initial prompt text, expanding the entity and / or the keyword by a pre-training model to obtain an extended prompt text, and obtaining an answer text generated by the pre-training model according to the extended prompt text. The answer text is an answer obtained by the pre-training model to a target question in the extended prompt text; a new extended prompt text is obtained according to the answer text, a standard answer corresponding to the target question, the extended prompt text, the entity and / or the keyword, and the pre-training model, until the similarity between the answer text generated by the pre-training model and the standard answer meets a preset condition. By automatically extracting the entity and / or the keyword, the pre-training model is used to automatically generate the extended prompt text, and whether to continue to expand the prompt text is determined according to the similarity between the answer text generated by the pre-training model and the standard answer, so that the final extended prompt text is more comprehensive and accurate, and more fully describes the user's demand and intention, and then guides the model to generate more accurate and valuable text. In this way, the generation efficiency of the prompt text is improved, the cost is reduced, and different user demands and field changes can be adapted.

[0072] In the embodiment, a general template can be pre-set to implement Prompt expansion in various scenarios. Based on this, please refer to Figure 2 The above step S102 can include:

[0073] In substep S1021, the first prompt text is generated according to the first preset prompt template and the entity and / or keyword.

[0074] It can be understood that the first preset prompt template is a pre-set general template, and the electronic device can generate the prompt text Prompt2 (i.e., the first prompt text) for guiding the model to expand the entity and keyword according to the first preset prompt template after extracting the entity and / or keyword in the initial prompt text. For example, Prompt2 can include the following content: "Please write more detailed Prompt according to the following keywords or entities".

[0075] In substep S1022, the first prompt text is input into the pre-trained model, the pre-trained model is guided to expand the entity and / or keyword by the first prompt text, and an expanded prompt text is output.

[0076] In this embodiment, the electronic device inputs the first prompt text Prompt2 into the pre-trained model by calling the API interface corresponding to the pre-trained model, the pre-trained model expands the entity and / or keyword under the guidance of Prompt2, and outputs the expanded prompt text Prompt_expand.

[0077] In actual application, considering that multiple standard answers may be set in some scenarios, such as in the digital human game commentary scenario, it is necessary to make a comment on the poor game of the anchor or to praise the good game of the anchor, so there may be various and various expression methods, thereby enriching the content of the commentary and avoiding too single commentary content. In order to enable the pre-trained model to output rich answer texts based on the expanded prompt text, the pre-trained model is caused to answer multiple times based on the expanded prompt text in this embodiment, so as to facilitate subsequent difference judgment of the answer text and the standard answer. Please refer to Figure 3 The above step S103 can include:

[0078] In substep S1031, the number of standard answers corresponding to the target question in the expanded prompt text is determined.

[0079] In substep S1032, the expanded prompt text is input into the pre-trained model multiple times according to the number of standard answers, the pre-trained model is guided to answer the target question in the expanded prompt text by the expanded prompt text, and answer texts output by the pre-trained model each time are obtained; wherein the number of answer texts is the same as the number of standard answers.

[0080] In this embodiment, the electronic device can determine the number of times that the pre-trained model answers the target question based on the extended prompt text according to the number of standard answers corresponding to the target question in the extended prompt text. After inputting the extended prompt text into the pre-trained model each time, the pre-trained model answers the target question under the guidance of the extended prompt text and outputs corresponding answer text. By inputting the extended prompt text multiple times, the answer texts obtained by the pre-trained model answering multiple times can be obtained.

[0081] In one example, the electronic device can input the extended prompt text into the pre-trained model to obtain the answer text, which can be implemented in the following manner: generating a prompt text Prompt3 for guiding the model to answer the target question according to the extended prompt text based on the extended prompt text and a preset second general template, and inputting Prompt3 into the pre-trained model to obtain the answer text of the target question by answering the target question according to the extended prompt text Prompt_expand under the guidance of Prompt3.

[0082] In one embodiment, the above sub-step S1031 specifically includes: searching for a target preset question matching the target question in the extended prompt text in a preset database, taking all standard answers corresponding to the target preset question as the standard answers corresponding to the target question, and obtaining the number of standard answers corresponding to the target question; the preset database stores a plurality of preset questions and standard answers corresponding to each preset question.

[0083] In this embodiment, the corresponding question and the corresponding multiple standard answers can be set in advance for different events, so as to obtain different preset questions and the multiple standard answers corresponding to the preset questions, and an offline database (i.e., a preset database) is constructed according to the multiple preset questions and the standard answers corresponding to each preset question. After obtaining the extended prompt text, the electronic device queries the preset database according to the target question in the extended prompt text, finds a target preset question matching (consistent or similar to a set threshold) the target question, and obtains all standard answers corresponding to the target preset question. Since the target preset question is consistent with or similar to the target question, the standard answers corresponding to the target preset question can be considered as the standard answers corresponding to the target question, and the number of standard answers corresponding to the target question can be obtained according to all standard answers corresponding to the target preset question.

[0084] Optionally, since the pre-trained model answers the target question based on the extended prompt text multiple times, the answer text obtained each time is compared with the standard answers to determine whether the current extended prompt text can make the pre-trained model output rich answers to meet the needs and intentions of the user. Based on this, please refer to Figure 4 The above step S104 can include:

[0085] Sub-step S1041, after obtaining each output answer text of the pre-trained model, the similarity scores between the answer text and each standard answer corresponding to the target question are calculated, and the highest similarity score is determined from each similarity score corresponding to the answer text.

[0086] In this embodiment, the electronic device can calculate the similarity scores between each answer text output by the pre-trained model and each standard answer, and select the highest similarity score therefrom, and then input the extended prompt text into the pre-trained model, and calculate the similarity scores based on the answer text obtained again and select the highest similarity score. The electronic device can also input the extended prompt text into the pre-trained model multiple times according to the number of standard answers, and after obtaining the answer texts generated by the pre-trained model in response to the target question multiple times, calculate the similarity scores between each of the multiple answer texts and each standard answer, and select the highest similarity score therefrom. The similarity score can represent the difference between the answer text and the standard answer, and the higher the score, the higher the similarity; the lower the score, the greater the difference.

[0087] For example, the number of standard answers corresponding to the target question Q is 10, which are A1, A2, A3, A4, A5, A6, A7, A8, A9, and A10. After the pre-trained model answers the target question Q for 10 times, the 10 answer texts obtained are A x1 , A x2 , A x3 , A x4 , A x5 , A x6 , A x7 , A x8 , A x9 , and A x10 . Assuming that the similarity scores between the answer text A x1 and the standard answers A1, A2, A3, A4, A5, A6, A7, A8, A9, and A10 are calculated, the similarity score between A x1 and A4 is the highest, which is 0.05; similarly, the similarity score between A x2 and A1 is the highest, which is 0.1; the similarity score between A x3 and A5 is the highest, which is 0.5; the similarity score between A x4 and A6 is the highest, which is 0.6; the similarity score between A x5 and A8 is the highest, which is 0.4; the similarity score between A x6 and A3 is the highest, which is 0.45; the similarity score between A x7 and A9 is the highest, which is 0.7; and the similarity score between A x8The similarity score with A10 is the highest, which is 0.2; A x9 The similarity score with A2 is the highest, which is 0.45; A x10 The similarity score with A7 is the highest, which is 0.5.

[0088] In an embodiment, the electronic device can also calculate the similarity scores between the answer text and each standard answer by using a pre-trained model. For example, the electronic device generates a prompt text Prompt4 for guiding the model to calculate the similarity scores according to the answer text, each standard answer, and a preset third universal template, inputs the Prompt4 into the pre-trained model, and then the pre-trained model evaluates the similarity between the answer text and each standard answer and scores under the guidance of the Prompt4, thereby obtaining the similarity scores between the answer text corresponding to the target question and each standard answer.

[0089] In sub-step S1042, the lowest target similarity scores are selected from the highest similarity scores corresponding to each answer text according to a preset proportion; the number of selected target similarity scores is determined according to the preset proportion and the number of answer texts.

[0090] In this embodiment, the preset proportion can be set according to actual needs, for example, it can be set to 10%, 20%, 30%, etc. Taking 20% as an example, since there are 10 answer texts, the lowest 2 target similarity scores are selected from the highest similarity scores corresponding to the 10 answer texts, i.e., A x1 The similarity score with A4 is 0.05; A x2 The similarity score with A1 is 0.1.

[0091] In sub-step S1043, if there is at least one target similarity score less than a preset threshold, the target answer text and the target standard answer corresponding to the target similarity score less than the preset threshold are obtained, a new extended prompt text is generated by the pre-trained model based on the target answer text, the target standard answer, the entity and / or the keyword, and the extended prompt text, until all target similarity scores selected according to the answer text generated by the pre-trained model are greater than or equal to the preset threshold.

[0092] In this embodiment, the preset threshold can be set according to actual needs, for example, it can be set to 0.5, 0.6, 0.7, etc. Taking 0.5 as an example, since the selected two target similarity scores 0.05 and 0.1 are both less than 0.5, it indicates that the answer text generated by the pre-trained model based on the current extended prompt text is not accurate enough, and the difference with the standard answer is still large, so the extended prompt text needs to be adjusted, and then the target answer text A x1, the target answer text A corresponding to the target standard answer A4 and the target similarity score 0.1 x2 , the target standard answer A1.

[0093] by the pre-training model based on the target answer text A x1 and the corresponding target standard answer A4, the target answer text A x2 and the corresponding target standard answer A1, the entity and / or keyword in the initial prompt text and the current extended prompt text, adjust the current extended prompt text to solve the difference between A x1 and A4, A x2 and A1, so as to obtain a new extended prompt text, and the pre-training model answers the target question multiple times according to the new extended prompt text. According to the obtained answer text, repeat the above sub-steps S1041-S1043 to realize the continuous adjustment of the extended prompt text. In this way, the answer text generated by the pre-training model according to the extended prompt text will gradually approach the standard answer, until all the target similarity scores selected are greater than or equal to the preset threshold. At this time, it is considered that the similarity between the answer text generated by the pre-training model and the standard answer reaches the preset condition.

[0094] In an embodiment, the above sub-step S1043 of generating a new extended prompt text by the pre-training model based on the target answer text, the target standard answer, the entity and / or keyword and the extended prompt text can specifically include: generating sample data according to the target answer text, the target standard answer and the target question, and generating a second prompt text according to the sample data, the entity and / or keyword, the extended prompt text and a second preset prompt template; inputting the second prompt text into the pre-training model, and guiding the pre-training model to adjust the extended prompt text based on the entity and / or keyword and the sample data according to the second prompt text, and outputting a new extended prompt text.

[0095] That is to say, since the similarity score between the target answer text and the target standard answer is less than the preset threshold, it indicates that the pre-training model cannot understand the content of the target standard answer based on the current extended prompt text. Therefore, sample data (few shot) is generated according to the target answer text, the target standard answer and the target question, and a second prompt text Prompt5 is generated according to the sample data, the previously extracted entity and / or keyword, the current extended prompt text and a second preset prompt template. By calling the API interface corresponding to the pre-training model, Prompt5 is input into the pre-training model. The pre-training model adjusts the extended prompt text according to the previously extracted entity and / or keyword and the few shot to be understood under the guidance of Prompt5, and obtains a new extended prompt text.

[0096] It can be understood that the second preset prompt template is also a general template, and the main difference between the second prompt text Prompt5 generated according to the second preset prompt template and the first prompt text Prompt2 generated according to the first preset prompt template is that the second prompt text Prompt5 not only provides the pre-training model with the keyword and / or entity content, but also provides the few shot and the current extended prompt text. The few shot can help the pre-training model better understand the task it needs to complete, and the pre-training model is adjusted based on the current extended prompt text, and finally the extended prompt text meeting the user's demand is obtained.

[0097] It should be noted that the prompt text extension method provided by the embodiments of the present application can be applied not only in the digital human game commentary scene, but also in many other scenes, and the embodiments of the present application do not limit this. For example, the following several possible scene examples:

[0098] Game live streaming classification: users can input game names or keywords to search for related game live streams, but sometimes users do not know the game names or keywords and just want to find live streams related to a certain game type or style. At this time, Prompt automatic expansion can be used to solve this problem. For example, the user can input "I want to watch a game live stream similar to the magic style", and the system can automatically expand the Prompt and generate a more specific Prompt such as "Recommend a game live stream similar to the magic style", thereby better meeting the user's demand.

[0099] Gift recommendation: there are many gifts that can be given to anchors in some live streaming applications, but sometimes users may not know what gift to give. At this time, Prompt automatic expansion can be used to provide more specific gift recommendations. For example, the user can input "I want to give an anchor a creative gift", and the system can automatically expand the Prompt and generate a more specific Prompt such as "Creative gift recommendation", thereby better meeting the user's demand.

[0100] Live room theme recommendation: some users may choose to follow or not follow an anchor because of the anchor's live room theme. For example, users may prefer to watch music-related live streams and not prefer to watch e-sports-related live streams. At this time, Prompt automatic expansion can be used to recommend anchors and live rooms that better meet users' interests. For example, the user can input "I want to watch a music-related live stream", and the system can automatically expand the Prompt and generate a more specific Prompt such as "Music live room recommendation", thereby better meeting the user's demand.

[0101] In order to perform the corresponding steps in the above embodiments and various possible manners, an implementation manner of a prompt text extension device is given as follows. Please refer toFigure 5 A functional module diagram of the prompt text expansion device 600 provided by the embodiment of the present application is shown. It should be noted that the prompt text expansion device 600 provided by the embodiment has the same basic principle and technical effects as the above-mentioned embodiments. For brief description, the part not mentioned in the embodiment can be referred to the corresponding content in the above-mentioned embodiments. The prompt text expansion device 600 comprises an extraction module 610, an expansion module 620, an answer text acquisition module 630 and a prompt text adjustment module 640.

[0102] The extraction module 610 is configured to extract entities and / or keywords in the initial prompt text.

[0103] It can be understood that the extraction module 610 can perform the above-mentioned step S101.

[0104] The expansion module 620 is configured to expand the entities and / or keywords by the pre-trained model to obtain an expanded prompt text.

[0105] It can be understood that the expansion module 620 can perform the above-mentioned step S102.

[0106] The answer text acquisition module 630 is configured to acquire an answer text generated by the pre-trained model according to the expanded prompt text; the answer text is an answer obtained by the pre-trained model to the target question in the expanded prompt text.

[0107] It can be understood that the answer text acquisition module 630 can perform the above-mentioned step S103.

[0108] The prompt text adjustment module 640 is configured to obtain a new expanded prompt text according to the answer text, the standard answer corresponding to the target question, the expanded prompt text, the entities and / or keywords and the pre-trained model, until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition.

[0109] It can be understood that the prompt text adjustment module 640 can perform the above-mentioned step S104.

[0110] Optionally, the expansion module 620 can be configured to generate a first prompt text according to a first preset prompt template and the entities and / or keywords; input the first prompt text into the pre-trained model, guide the pre-trained model to expand the entities and / or keywords by the first prompt text, and output the expanded prompt text.

[0111] It can be understood that the expansion module 620 can perform the above-mentioned sub-steps S1021 and S1022.

[0112] Optionally, the answer text obtaining module 630 can be configured to determine the number of standard answers corresponding to the target question in the extended prompt text, input the extended prompt text into the pre-trained model for multiple times according to the number of standard answers, guide the pre-trained model to answer the target question in the extended prompt text through the extended prompt text, and obtain the answer texts output by the pre-trained model each time; wherein the number of answer texts is the same as the number of standard answers.

[0113] It can be understood that the answer text obtaining module 630 can perform the above-mentioned sub-steps S1031 and S1032.

[0114] Optionally, the answer text obtaining module 630 is specifically configured to find a target preset question matching the target question in the extended prompt text in a preset database, take all standard answers corresponding to the target preset question as the standard answers corresponding to the target question, and obtain the number of standard answers corresponding to the target question; the preset database stores a plurality of preset questions and the standard answers corresponding to each preset question.

[0115] Optionally, the prompt text adjusting module 640 can be configured to calculate the similarity scores between the answer texts and the standard answers corresponding to the target question after obtaining the answer texts output by the pre-trained model each time, and determine the highest similarity scores from the similarity scores corresponding to the answer texts; wherein each answer text corresponds to one highest similarity score; select the lowest target similarity scores from the highest similarity scores corresponding to the answer texts according to a preset proportion; the number of target similarity scores is determined according to the preset proportion and the number of answer texts; if there is at least one target similarity score less than a preset threshold, obtain the target answer text and the target standard answer corresponding to the target similarity score less than the preset threshold, generate a new extended prompt text through the pre-trained model based on the target answer text, the target standard answer, the entity and / or the keyword, and the extended prompt text, until all target similarity scores selected from the answer texts generated by the pre-trained model are greater than or equal to the preset threshold.

[0116] It can be understood that the prompt text adjusting module 640 can perform the above-mentioned steps S1041 to S1043.

[0117] Optionally, the prompt text adjusting module 640 is specifically configured to generate sample data according to the target answer text, the target standard answer, and the target question, and generate a second prompt text according to the sample data, the entity and / or the keyword, the extended prompt text, and a second preset prompt template; input the second prompt text into the pre-trained model, guide the pre-trained model to adjust the extended prompt text based on the entity and / or the keyword and the sample data through the second prompt text, and output a new extended prompt text.

[0118] It can be seen that the prompt text expansion device provided by the embodiment of the present application comprises an extraction module, an expansion module, an answer text acquisition module and a prompt text adjustment module. The extraction module is used to extract entities and / or keywords in the initial prompt text. The expansion module is used to expand the entities and / or keywords through a pre-trained model to obtain an expanded prompt text. The answer text acquisition module is used to acquire an answer text generated by the pre-trained model according to the expanded prompt text. The answer text is an answer obtained by the pre-trained model in response to a target question in the expanded prompt text. The prompt text adjustment module is used to obtain a new expanded prompt text according to the answer text, a standard answer corresponding to the target question, the expanded prompt text, the entities and / or keywords and the pre-trained model until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition. By automatically extracting the entities and / or keywords, the expanded prompt text is automatically generated based on the pre-trained model, and it is judged whether to continue the prompt text expansion according to the similarity between the answer text generated by the pre-trained model and the standard answer, so that the expanded prompt text obtained finally is more comprehensive and accurate, and more fully describes the needs and intentions of the user, thereby guiding the model to generate more accurate and more valuable text. In this way, the generation efficiency of the prompt text is improved, the cost is reduced, and different user needs and field changes can be adapted.

[0119] Please refer to Figure 6 A block schematic diagram of an electronic device 100 provided by the embodiment of the present application is shown. The electronic device 100 comprises a memory 110, a processor 120 and a communication module 130. The memory 110, the processor 120 and the communication module 130 are directly or indirectly electrically connected to each other to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0120] The memory 110 is used to store programs or data. The memory 110 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM) and the like.

[0121] The processor 120 is configured to read / write data or programs stored in the memory 110, and perform corresponding functions. For example, when a computer program stored in the memory 110 is executed by the processor 120, the prompting text expansion method disclosed in the above embodiments can be implemented.

[0122] The communication module 130 is configured to establish a communication connection between the electronic device 100 and other devices through a network, and configured to transceive data through the network.

[0123] It should be understood that, Figure 6 The structure shown is only a structural schematic diagram of the electronic device 100, and the electronic device 100 can further include more or less components than those shown in the Figure 6 different configuration from that shown in the Figure 6 different configuration from that shown in the Figure 6 The components shown in the above embodiments can be implemented in hardware, software or a combination thereof.

[0124] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor 120 to implement the prompting text expansion method disclosed in the above embodiments.

[0125] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can also be implemented by other manners. The apparatus embodiments described above are only schematic, for example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logic function. It should also be noted that, in some alternative implementation manners, the functions noted in the blocks can occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for implementing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0126] In addition, each functional module in the embodiments of the present application can be integrated together to form a separate part, or each module can exist independently, or two or more modules can be integrated to form a separate part.

[0127] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0128] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for expanding prompt text, characterized in that, The method includes: Extract entities and / or keywords from the initial prompt text; The entities and / or keywords are expanded using a pre-trained model to obtain expanded prompt text; Obtain the answer text generated by the pre-trained model based on the extended prompt text; the answer text is the answer obtained by the pre-trained model in answering the target question in the extended prompt text; Based on the answer text, the standard answer corresponding to the target question, the extended hint text, the entities and / or keywords, and the pre-trained model, new extended hint text is obtained until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition, including: After obtaining the answer text output by the pre-trained model each time, the similarity score between the answer text and each standard answer corresponding to the target question is calculated, and the highest similarity score is determined from the similarity scores corresponding to the answer text. The lowest target similarity score is selected from the highest similarity scores corresponding to each answer text according to a preset ratio; If at least one of the target similarity scores is less than a preset threshold, then the target answer text and target standard answer corresponding to the target similarity scores less than the preset threshold are obtained. The pre-trained model generates new extended prompt text based on the target answer text, the target standard answer, the entity and / or keywords, and the extended prompt text, until all target similarity scores selected based on the answer text generated by the pre-trained model are greater than or equal to the preset threshold.

2. The method for expanding prompt text according to claim 1, characterized in that, The step of expanding the entities and / or keywords using a pre-trained model to obtain expanded prompt text includes: Generate a first prompt text based on the first preset prompt template and the entity and / or keywords; The first prompt text is input into the pre-trained model, which is then guided by the first prompt text to expand the entity and / or keyword, and outputs the expanded prompt text.

3. The method for expanding prompt text according to claim 1, characterized in that, The step of obtaining the answer text generated by the pre-trained model based on the expanded prompt text includes: Determine the number of standard answers corresponding to the target question in the extended prompt text; The extended prompt text is input into the pre-trained model multiple times according to the number of standard answers. The extended prompt text guides the pre-trained model to answer the target question in the extended prompt text, and the answer text output by the pre-trained model each time is obtained; wherein, the number of answer texts is the same as the number of standard answers.

4. The method for expanding prompt text according to claim 3, characterized in that, Each answer text corresponds to a highest similarity score; the number of target similarity scores selected is determined according to the preset ratio and the number of answer texts.

5. The method for expanding prompt text according to claim 4, characterized in that, The step of generating new extended hint text based on the target answer text, the target standard answer, the entities and / or keywords, and the extended hint text using the pre-trained model includes: Sample data is generated based on the target answer text, the target standard answer, and the target question. A second prompt text is generated based on the sample data, the entity and / or keywords, the extended prompt text, and the second preset prompt template. The second prompt text is input into the pre-trained model, and the pre-trained model is guided by the second prompt text to adjust the extended prompt text based on the entity and / or keywords and the sample data, and output a new extended prompt text.

6. The method for expanding prompt text according to claim 3, characterized in that, Determining the number of standard answers corresponding to the target question in the extended prompt text includes: The system searches a preset database for a target preset question that matches the target question in the extended prompt text, takes all the standard answers corresponding to the target preset question as the standard answer corresponding to the target question, and obtains the number of standard answers corresponding to the target question. The preset database stores multiple preset questions and the standard answer corresponding to each preset question.

7. A prompt text expansion device, characterized in that, The device includes: The extraction module is used to extract entities and / or keywords from the initial prompt text; An extension module is used to extend the entities and / or keywords using a pre-trained model to obtain extended prompt text; The answer text acquisition module is used to acquire the answer text generated by the pre-trained model based on the extended prompt text; the answer text is the answer obtained by the pre-trained model in answering the target question in the extended prompt text. The prompt text adjustment module is used to obtain new extended prompt text based on the answer text, the standard answer corresponding to the target question, the extended prompt text, the entity and / or keywords, and the pre-trained model, until the similarity between the answer text generated by the pre-trained model and the standard answer reaches a preset condition; The prompt text adjustment module is used to calculate the similarity score between the answer text and each standard answer corresponding to the target question after obtaining the answer text output by the pre-trained model each time, and determine the highest similarity score from the similarity scores corresponding to the answer text; select the lowest target similarity score from the highest similarity scores corresponding to each answer text according to a preset ratio; if there is at least one target similarity score less than a preset threshold, obtain the target answer text and target standard answer corresponding to the target similarity score less than the preset threshold, and generate a new extended prompt text based on the target answer text, the target standard answer, the entity and / or keywords, and the extended prompt text by the pre-trained model, until all target similarity scores selected based on the answer text generated by the pre-trained model are greater than or equal to the preset threshold.

8. The prompt text expansion device according to claim 7, characterized in that, The answer text acquisition module is used to determine the number of standard answers corresponding to the target question in the extended prompt text; input the extended prompt text multiple times into the pre-trained model according to the number of standard answers, guide the pre-trained model to answer the target question in the extended prompt text through the extended prompt text, and obtain the answer text output by the pre-trained model each time; wherein, the number of answer texts is the same as the number of standard answers; each answer text corresponds to a highest similarity score; the number of target similarity scores selected is determined according to the preset ratio and the number of answer texts.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the prompt text expansion method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the prompt text expansion method as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Test data processing method and device for question-answering system and terminal

    CN106599317A

  • Method and device for knowledge quiz

    CN106776532A