Question content generation method and device, electronic equipment and storage medium

Abstract

The invention provides a questioning content generation method and device, electronic equipment and a storage medium, and relates to the technical field of AI interview. According to the method, when the answer content of a to-be-asked object meets a preset asking condition, an asking transition language is generated according to the answer content, and a plurality of candidate asking trunk contents and the asking probability of each candidate asking trunk content are obtained based on the answer content and the asking content corresponding to the answer content; according to the method and the device, the target query trunk content is determined from all the candidate query trunk contents according to the query probability of each candidate query trunk content, and the target query trunk content and the query transition language are utilized to obtain the target query content, so that the quality of the query content is ensured, and the query diversity is provided.

Description

Technical Field

[0001] The present invention relates to the field of AI interview technology, and in particular to a method, device, electronic device and storage medium for generating follow-up question content. Background Art

[0002] Traditional AI interview follow-up questions are mainly based on discriminative models represented by BERT. According to the interview topic and the user's answer, the discriminative model is used to select the most suitable question from the existing follow-up question library for follow-up questions.

[0003] The traditional method is limited by the follow-up question bank, which is a finite set. When compiling the follow-up question bank, it is necessary to make a lot of predictions about the candidates' answers and add follow-up questions based on the predicted candidates' answers. Therefore, the compilation process of the follow-up question bank is greatly affected by the ability of the compiler, and the quality often fluctuates greatly, which will affect the diversity of the follow-up questions. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for generating follow-up question content, which can ensure the quality of the follow-up question content.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides a method for generating question content, the method comprising:

[0007] When the answer of the subject to be questioned meets the preset questioning condition, generating a questioning transitional phrase according to the answer;

[0008] Based on the answer content and the question content corresponding to the answer content, a plurality of candidate follow-up question main contents and a follow-up question probability of each candidate follow-up question main content are obtained;

[0009] Determining a target question trunk content from all candidate question trunk contents according to the question probability of each candidate question trunk content;

[0010] The target question content is obtained by using the target question main content and the question transition words.

[0011] Optionally, the method further includes a step of determining whether the answer content satisfies a preset question condition, which includes:

[0012] Inputting the answer content into a pre-trained first classification model, and determining whether the answer category to which the answer content belongs can be obtained according to the output of the first classification model, wherein the first classification model is obtained by fine-tuning a preset discriminant model;

[0013] If yes, judging whether the answer content satisfies the preset follow-up question condition according to the answer category to which the answer content is obtained;

[0014] If not, the answer content is input into a pre-trained second classification model to obtain the answer category to which the answer content belongs, and based on the obtained answer category to which the answer content belongs, it is determined whether the answer content meets the preset follow-up question conditions, wherein the second classification model is obtained by fine-tuning the pre-trained large language model for preset classification prompt words.

[0015] Optionally, the step of determining whether the answer content satisfies a preset follow-up question condition according to the answer category to which the answer content belongs includes:

[0016] If the answer category is a preset category, determining that the answer content meets the preset follow-up question condition;

[0017] If the answer category is not a preset category, it is determined that the answer content does not meet the preset follow-up question condition.

[0018] Optionally, the step of generating a follow-up question transitional phrase according to the answer content includes:

[0019] Extracting summary text from the answer content using a pre-trained summary extraction model, wherein the summary extraction model is obtained by fine-tuning a pre-trained large language model for preset summary extraction prompt words;

[0020] The summary text is used as the question transitional words.

[0021] Optionally, the candidate follow-up question main content includes a first candidate follow-up question main content and a second candidate follow-up question main content, and the step of obtaining a plurality of candidate follow-up question main contents and a follow-up question probability of each candidate follow-up question main content based on the answer content and the question content corresponding to the answer content includes:

[0022] Inputting the answer content, the question content and the preset follow-up question strategy into a pre-trained follow-up question content generation model to obtain at least one of the first candidate follow-up question trunk contents, wherein the follow-up question content generation model is obtained by fine-tuning the pre-trained large language model for preset follow-up question prompt words;

[0023] All of the first candidate follow-up question main contents and a plurality of preset second candidate follow-up question main contents are input into a pre-trained prediction model to obtain the questioning probability of each of the first candidate follow-up question main contents and each of the second candidate follow-up question main contents.

[0024] Optionally, the method further comprises:

[0025] The questioning probability of each of the first candidate questioning main contents is increased by a preset value, and the questioning probability of each of the second candidate questioning main contents is kept unchanged, and the step of determining the target questioning main content from all the candidate questioning main contents according to the questioning probability of each of the candidate questioning main contents is performed.

[0026] Optionally, the step of determining the target question main content from all candidate question main contents according to the questioning probability of each candidate question main content comprises:

[0027] The candidate question main content with the highest questioning probability among all the candidate question main content is used as the target question main content.

[0028] In a second aspect, the present invention provides a device for generating question content, the device comprising:

[0029] A generating module, for generating a transitional phrase for further questioning according to the answer content of the subject to be further questioned when the answer content of the subject to be further questioned meets the preset further questioning condition;

[0030] An obtaining module, configured to obtain a plurality of candidate follow-up question trunk contents and a follow-up question probability of each candidate follow-up question trunk content based on the answer content and the question content corresponding to the answer content;

[0031] The determination module is used to determine the target question main content from all the candidate question main content according to the questioning probability of each candidate question main content; and obtain the target question content by using the target question main content and the question transition words.

[0032] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for generating follow-up content as described in the first aspect is implemented.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method for generating follow-up content as described in the first aspect.

[0034] Compared with the prior art, the method for generating follow-up question content provided by the present invention is as follows: when the answer content of the object to be questioned meets the preset questioning condition, a follow-up question transition word is generated according to the answer content; based on the answer content and the question content corresponding to the answer content, multiple candidate follow-up question main contents and the questioning probability of each candidate follow-up question main content are obtained; according to the questioning probability of each candidate follow-up question main content, the target follow-up question main content is determined from all candidate follow-up question main contents; the target follow-up question main content and the follow-up question transition word are used to obtain the target follow-up question content. Since the embodiment of the present invention generates the follow-up question transition word according to the answer content that meets the preset questioning condition, multiple candidate follow-up question main contents are obtained based on the answer content and the corresponding question content, and then the target follow-up question main content is determined according to the questioning probability of each candidate follow-up question main content, and the target follow-up question content is obtained by using the target follow-up question main content and the follow-up question transition word, the quality of the follow-up question content is guaranteed and the diversity of follow-up questions is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 A schematic diagram of a flow chart of a method for generating follow-up question content provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of a method for determining whether an answer content satisfies a preset follow-up question condition provided by an embodiment of the present invention;

[0038] Figure 3 A functional unit block diagram of a device for generating question content provided by an embodiment of the present invention;

[0039] Figure 4 A schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0040] Icons: 100 - question content generating device; 101 - generating module; 102 - obtaining module; 103 - determining module; 200 - electronic device; 210 - memory; 220 - processor. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0042] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0044] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0045] In order to ensure the quality of follow-up question content and improve the diversity of follow-up question content, the present invention provides a follow-up question content generation method, which will be described in detail below.

[0046] Please refer to Figure 1 The method for generating follow-up question content includes steps S101 to S104.

[0047] S101, when the answer of the subject to be questioned meets the preset questioning conditions, generate a questioning transitional phrase according to the answer.

[0048] Among them, the preset follow-up question condition means that the answer category of the answer content is a preset category, and the preset categories include normal answer, negative answer and irrelevant answer. That is, if the answer category to which the answer content belongs belongs to any of the normal answer, negative answer and irrelevant answer, then it is determined that the answer content meets the preset follow-up question condition.

[0049] When the answer category of the answer of the subject to be questioned is "normal answer", a summary of the answer is extracted and the obtained summary text is used as a transitional word for questioning.

[0050] Furthermore, a pre-trained summary extraction model can be used to extract summary text from the answer content.

[0051] The pre-trained summary extraction model is obtained by fine-tuning the pre-trained large language model for preset summary extraction prompt words.

[0052] The pre-trained large language model may be LLAMA, ChatGLM, or Qwen, etc. The input layer of the pre-trained large language model introduces pre-written summary extraction prompt words to guide the pre-trained large language model to obtain summary text by summarizing the answer content.

[0053] Since automatic speech recognition technology will inevitably convert some speech into erroneous text, when writing summary extraction prompt words, you can set them in the prompt words to allow the pre-trained large language model to perform text correction while extracting the summary.

[0054] Exemplarily, the abstract extraction prompt words may be "Please read the following paragraphs carefully, and while creating the abstract, pay attention to correcting any factual errors or grammatical problems in the text to ensure the accuracy of the abstract", "Extract the main information from the following text, and pay attention to correcting any inaccurate descriptions or data to produce an accurate abstract", or "When summarizing the following content, please check and correct any obvious errors, including spelling, grammatical or data errors to maintain the accuracy of the abstract".

[0055] When the answer category of the subject to be questioned is "negative answer" or "irrelevant answer", the preset fixed reminder text is directly used as the transitional words for the question to tell the subject to be questioned that his answer does not meet expectations.

[0056] Next, the implementation method of "determining whether the answer content meets the preset follow-up question conditions" is introduced.

[0057] Please refer to Figure 2 , the implementation method includes steps S201 to S203.

[0058] S201, input the answer content into a pre-trained first classification model, and determine whether the answer category to which the answer content belongs can be obtained based on the output of the first classification model.

[0059] The first classification model is obtained by training a preset discriminant model. The preset discriminant model can be BERT, TextCNN, etc. The first classification model can directly output the probability that the answer content belongs to each candidate category. Each candidate category includes normal answer, negative answer, irrelevant answer, refusal to answer, moral incompatibility, etc.

[0060] In a possible implementation, if there is a candidate category with a probability greater than 90% in the output of the first classification model, the candidate category can be used as the answer category to which the answer content belongs. It can be understood that if there is no candidate category with a probability greater than 90% in the output of the first classification model, the answer category to which the answer content belongs cannot be determined.

[0061] In an embodiment of the present invention, if the answer category to which the answer content belongs can be obtained based on the output of the first classification model, step S202 is executed; if the answer category to which the answer content belongs cannot be obtained based on the output of the first classification model, step S203 is executed.

[0062] S202: judging whether the answer content satisfies a preset follow-up question condition according to the answer category to which the answer content is obtained.

[0063] S203, input the answer content into the pre-trained second classification model, obtain the answer category to which the answer content belongs, and judge whether the answer content meets the preset follow-up question condition according to the obtained answer category to which the answer content belongs.

[0064] Among them, the second classification model is obtained by fine-tuning the pre-trained large language model for preset classification prompt words.

[0065] The pre-trained large language model can be LLAMA, ChatGLM or Qwen, etc. The input layer of the pre-trained large language model introduces pre-written classification prompt words to guide the pre-trained large language model to determine the answer category by understanding the answer content.

[0066] For example, the classification prompt words may be "Please analyze the following interview answers and classify them as normal answers, negative answers, irrelevant answers, refusal to answer, or moral inconsistency based on their contents", "Read the following interview answers and determine which of the following categories it belongs to: normal answers, negative answers, irrelevant answers, refusal to answer, or moral inconsistency" or "Analyze this interview answer and indicate which category it is more inclined to: normal answers, negative answers, irrelevant answers, refusal to answer, and moral inconsistency".

[0067] After determining the answer category to which the answer content belongs, judge whether the answer content meets the preset follow-up question conditions based on the answer category to which the answer content belongs. The corresponding implementation process may be: judge whether the answer category to which the answer content belongs is the preset category; if so, judge that the answer content meets the preset follow-up question conditions; if not, judge that the answer content does not meet the preset follow-up question conditions.

[0068] Since the preset categories include normal answers, negative answers and irrelevant answers, it can be understood that if the answer content belongs to any of the normal answers, negative answers and irrelevant answers, then the answer content is determined to meet the preset follow-up question conditions; if the answer content belongs to a refusal to answer or is immoral, then the answer content is determined not to meet the preset follow-up question conditions.

[0069] Next, continue Figure 1 Steps S102 to S103 in FIG.

[0070] S102, based on the answer content and the question content corresponding to the answer content, a plurality of candidate follow-up question main content and a follow-up question probability of each candidate follow-up question main content are obtained.

[0071] The candidate follow-up question main content includes a first candidate follow-up question main content and a second candidate follow-up question main content.

[0072] The first candidate follow-up question trunk content is obtained by inputting the answer content, the question content and the preset follow-up question strategy into a pre-trained follow-up question content generation model.

[0073] The follow-up question content generation model is obtained by fine-tuning the pre-trained large language model for preset follow-up question prompts. The pre-trained large language model is LLAMA, ChatGLM or Qwen, etc. The input layer of the pre-trained large language model introduces pre-written follow-up question prompts, and the pre-trained language model plays the role of an interviewer to ask questions through the follow-up question prompts.

[0074] The preset follow-up questioning strategy can be the "STAR interview method" often used in interviews, or other interview follow-up questioning guidelines.

[0075] For example, when the question is "What is the biggest setback you have ever experienced" and the answer is "Failure in the postgraduate entrance examination", and the preset follow-up question strategy is "In-depth understanding of the behaviors in the experience described by the candidate", then the follow-up content generation model will ask questions about specific behaviors in the postgraduate entrance examination, such as "Which behaviors of yours in the postgraduate entrance examination do you think led to your failure?" This way of generating questions does not require special writing for each question. It is an infinite set and can truly achieve "one thousand faces for one thousand people".

[0076] The main content of the second candidate follow-up questions is pre-set and is some excellent follow-up questions manually annotated by interview experts, such as "What are your reflections after experiencing this setback?" and "How do you avoid similar things from happening again next time?"

[0077] All first candidate follow-up question trunk contents and multiple second candidate follow-up question trunk contents are input into a pre-trained prediction model to obtain the question probability of each first candidate follow-up question trunk content and each second candidate follow-up question trunk content.

[0078] The prediction model may be obtained by training a discriminant model, and the question probability refers to the probability of being suitable for questioning.

[0079] S103, determining a target question main content from all candidate question main contents according to the questioning probability of each candidate question main content.

[0080] In a possible implementation, the candidate question trunk content with the highest questioning probability among all candidate question trunk contents may be used as the target question trunk content.

[0081] Furthermore, in order to give priority to the first candidate follow-up question main content being selected as the target follow-up question main content, in an embodiment of the present invention, the questioning probability of each first candidate follow-up question main content can be increased by a preset value, and the questioning probability of each second candidate follow-up question main content can be kept unchanged, and then the candidate follow-up question main content with the highest questioning probability among all the candidate follow-up question main contents can be selected as the target follow-up question main content.

[0082] The preset value may be set as required, for example, the preset value may be set to 15%, that is, the probability of asking questions about each first candidate question trunk content is increased by 15%.

[0083] S104, obtaining the target question content by using the target question main content and the question transition words.

[0084] After combining the main content of the target question and the transition words of the question, they are input into pre-trained language models such as LLAMA, ChatGLM or iQwen for polishing, supplementing the content of the question stem, and improving the connection between the obtained target question content and the answer content of the person to be questioned.

[0085] For example, when the interview question asks the candidate to describe an experience of solving a problem, and the candidate's answer mentions a method of solving the problem, and the main content of the target question is "how do you evaluate that the problem has been solved", a possible polishing direction of the pre-trained language model is to quote the method mentioned by the candidate, ask the candidate how to evaluate that the problem has been solved, and ask the candidate what level of relevant indicators has been achieved based on the method provided by the candidate.

[0086] In order to execute the corresponding steps in the above method embodiment and various possible implementations, an implementation of the question content generating device 100 is provided below.

[0087] Please refer to Figure 3 The question content generating device 100 includes a generating module 101 , an obtaining module 102 and a determining module 103 .

[0088] The generating module 101 is used to generate a transitional phrase for further questioning according to the answer content of the subject to be further questioned when the answer content of the subject to be further questioned meets the preset further questioning condition.

[0089] The obtaining module 102 is used to obtain a plurality of candidate follow-up question main contents and a follow-up question probability of each candidate follow-up question main content based on the answer content and the question content corresponding to the answer content.

[0090] The determination module 103 is used to determine the target question main content from all candidate question main contents according to the question probability of each candidate question main content; and obtain the target question main content by using the target question main content and the question transition words.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described question content generating device 100 can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0092] Furthermore, the embodiment of the present invention also provides an electronic device 200, please refer to Figure 4 , the electronic device 200 may include a memory 210 and a processor 220 .

[0093] The processor 220 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the question content generation method provided in the above method embodiment.

[0094] The memory 210 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 210 may exist independently and be connected to the processor 220 via a communication bus. The memory 210 may also be integrated with the processor 220. Among them, the memory 210 is used to store machine executable instructions for executing the scheme of the present invention. The processor 220 is used to execute the machine executable instructions stored in the memory 210 to implement the above-mentioned method embodiment.

[0095] The embodiment of the present invention further provides a computer-readable storage medium including a computer program. When the computer program is executed, it can be used to perform the relevant operations in the method for generating question content provided by the above method embodiment.

[0096] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for generating follow-up question content, characterized in that: The method comprises: When the answer of the subject to be questioned meets the preset questioning condition, generating a questioning transitional phrase according to the answer; Based on the answer content and the question content corresponding to the answer content, a plurality of candidate follow-up question trunk contents and a follow-up question probability of each candidate follow-up question trunk content are obtained; Determining a target question trunk content from all candidate question trunk contents according to the questioning probability of each candidate question trunk content; The target question content is obtained by using the target question main content and the question transition words.

2. The method according to claim 1, characterized in that The method further includes a step of determining whether the answer content satisfies a preset question condition, which includes: Inputting the answer content into a pre-trained first classification model, and determining whether the answer category to which the answer content belongs can be obtained according to the output of the first classification model, wherein the first classification model is obtained by training a preset discriminant model; If yes, judging whether the answer content satisfies the preset follow-up question condition according to the answer category to which the answer content is obtained; If not, the answer content is input into a pre-trained second classification model to obtain the answer category to which the answer content belongs, and based on the obtained answer category to which the answer content belongs, it is determined whether the answer content meets the preset follow-up question conditions, wherein the second classification model is obtained by fine-tuning the pre-trained large language model for preset classification prompt words.

3. The method according to claim 2, characterized in that The step of judging whether the answer content satisfies a preset question condition according to the answer category to which the answer content belongs comprises: If the answer category is a preset category, determining that the answer content meets the preset follow-up question condition; If the answer category is not a preset category, it is determined that the answer content does not meet the preset follow-up question condition.

4. The method according to claim 1, characterized in that The step of generating a follow-up question transitional phrase according to the answer content comprises: Extracting summary text from the answer content using a pre-trained summary extraction model, wherein the summary extraction model is obtained by fine-tuning a pre-trained large language model for preset summary extraction prompt words; The summary text is used as the question transitional words.

5. The method according to claim 1, characterized in that The candidate follow-up question main content includes a first candidate follow-up question main content and a second candidate follow-up question main content. The step of obtaining a plurality of candidate follow-up question main contents and a follow-up question probability of each candidate follow-up question main content based on the answer content and the question content corresponding to the answer content includes: Inputting the answer content, the question content and the preset follow-up question strategy into a pre-trained follow-up question content generation model to obtain at least one of the first candidate follow-up question trunk contents, wherein the follow-up question content generation model is obtained by fine-tuning the pre-trained large language model for preset follow-up question prompt words; All of the first candidate follow-up question main contents and a plurality of preset second candidate follow-up question main contents are input into a pre-trained prediction model to obtain the questioning probability of each of the first candidate follow-up question main contents and each of the second candidate follow-up question main contents.

6. The method according to claim 5, characterized in that The method further comprises: The questioning probability of each of the first candidate questioning main contents is increased by a preset value, and the questioning probability of each of the second candidate questioning main contents is kept unchanged, and the step of determining the target questioning main content from all the candidate questioning main contents according to the questioning probability of each of the candidate questioning main contents is performed.

7. The method according to claim 1, characterized in that The step of determining the target question trunk content from all candidate question trunk contents according to the question probability of each candidate question trunk content comprises: The candidate question main content with the highest questioning probability among all the candidate question main content is used as the target question main content.

8. A device for generating question content, characterized in that: The device comprises: A generating module, for generating a transitional phrase for further questioning according to the answer content of the subject to be further questioned when the answer content of the subject to be further questioned meets the preset further questioning condition; An obtaining module, configured to obtain a plurality of candidate follow-up question trunk contents and a follow-up question probability of each candidate follow-up question trunk content based on the answer content and the question content corresponding to the answer content; The determination module is used to determine the target question main content from all the candidate question main content according to the questioning probability of each candidate question main content; and obtain the target question content by using the target question main content and the question transition words.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the method for generating question content according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the method for generating question content as described in any one of claims 1 to 7.

Images