Interview coaching method and device based on large language model, equipment and medium

Abstract

The application relates to the technical field of deep learning, in particular to an interview guidance method and device based on a large language model, equipment and a medium. The method comprises the following steps: obtaining resume data of a job seeker and target post demand data corresponding to the resume data; inputting the resume data of the job seeker and the target post demand data into a large language model and outputting a first interview question and a corresponding first interview answer; inputting the first interview question and the first interview answer into a relevancy scoring model and outputting a relevancy score; inputting the resume data of the job seeker and the first interview answer into a credibility scoring model and outputting a credibility score; if a comprehensive score is less than a preset threshold, inputting the relevancy score and the credibility score into the large language model again until the comprehensive score is greater than the preset threshold, and outputting the first interview question and the corresponding interview answer. The technical scheme of the application can effectively improve the quality of the finally output answer and help the job seeker improve the interview performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to an interview coaching method and device based on a large language model, an electronic device and a storage medium. BACKGROUND

[0002] In the modern job-hunting process, interviews have become a key link that determines whether a job seeker can obtain a target position. Since interview performance directly affects the results of recruitment, many job seekers expect to improve their competitiveness through interview coaching services. Such services can help job seekers better demonstrate their personal advantages and highlight their strengths in light of job requirements, and thus have gradually been widely welcomed by job seekers.

[0003] Current interview coaching services mainly rely on manual services, which provide one-on-one customized coaching for job seekers through expert teams such as senior HRs, headhunters and career planners. Such services can deeply explore the personal ability advantages of job seekers and provide detailed interview skill guidance. However, due to the high labor cost of such services, the prices are usually expensive, resulting in a low market penetration rate. In addition, the limited nature of expert resources also restricts the large-scale popularization of such services, making it difficult to effectively meet the needs of a large number of job seekers.

[0004] On the other hand, the rapid development of artificial intelligence technology has given birth to a class of AI-based interview simulation systems. Such systems usually have the functions of automatically generating interview questions and evaluating interview performance, and can help improve interview efficiency to a certain extent. However, the main purpose of such systems is to improve the efficiency of recruitment screening and evaluation, and they are insufficient in exploring the advantages of job seekers' experiences, making it difficult to effectively provide personalized coaching suggestions for job seekers. SUMMARY

[0005] To solve the above problems, the application provides an interview tutoring method based on a large language model, which comprises the following steps: obtaining resume data of a job seeker and target job requirement data corresponding to the resume data; inputting the resume data of the job seeker and the target job requirement data into a large language model and outputting a first interview question and a corresponding first interview answer; inputting the first interview question and the first interview answer into a pre-trained relevance score model and outputting a relevance score, wherein the relevance score model is trained by using a plurality of first sample data, the first sample data comprises an interview question and an interview answer, and the first sample data is divided into positive samples and negative samples according to the relevance of the interview question and the interview answer; inputting the resume data of the job seeker and the first interview answer into a pre-trained credibility score model and outputting a credibility score, wherein the credibility score model is trained by using a plurality of second sample data, the second sample data comprises resume data and an interview answer, and the type of the second sample data is divided into: implication, contradiction and neutrality according to the logical relationship between the resume data and the interview answer; determining a comprehensive score according to the relevance score and the credibility score; if the comprehensive score is less than a preset threshold, inputting the relevance score and the credibility score into the large language model again to output a second interview answer, and calculating a comprehensive score of the first interview question and the second interview answer until the comprehensive score is greater than the preset threshold, and outputting the first interview question and the corresponding interview answer.

[0006] The interview tutoring method as described above, wherein the relevance score model is trained by using the relevance score of the same interview question with a relevant interview answer and an irrelevant interview answer.

[0007] The interview tutoring method as described above, wherein the training process of the relevance score model comprises the following steps: constructing a plurality of first sample data, wherein the positive samples comprise an interview question and an interview answer related to the interview question, and the negative samples comprise an interview question and an interview answer irrelevant to the interview question; indexing the interview question and the interview answer in the first sample data to obtain a first sample sequence code, wherein the first sample sequence code comprises a classification identifier for indicating whether the interview question and the interview answer are relevant; inputting the first sample sequence code into the relevance score model to output the relevance score; updating the model parameters in the relevance score model by minimizing a first loss function value until the first loss function value is less than a first loss threshold, wherein the first loss function comprises the relevance score of the positive samples and the relevance score of the negative samples.

[0008] The interview tutoring method as described above, wherein the first loss function is represented by the following formula:

[0009]

[0010] wherein lossA is the first loss function value, s θ (q,a+ ) is a correlation score of a positive sample, is a correlation score of a negative sample, and k is the number of negative samples.

[0011] The interview coaching method as described above, the training process of the credibility scoring model comprises: constructing a plurality of second sample data, the second sample data comprising resume data and interview answers; indexing the resume data and the interview answers in the second sample data to obtain second sample sequence encodings, the second sample sequence encodings comprising classification identifiers for representing logical relationships between the resume data and the interview answers; inputting the second sample sequence encodings into the credibility scoring model to output unnormalized prediction values on each logical relationship and convert the unnormalized prediction values into probability distribution values on each logical relationship; updating model parameters in the credibility scoring model by minimizing a second loss function value until the second loss function value is less than a second loss threshold, the second loss function comprising true types of the second sample data and the probability distribution values on each logical relationship.

[0012] The interview coaching method as described above, the second loss function is represented by the following formula:

[0013]

[0014] wherein lossB is the second loss function value, p c is a probability that the sample data belongs to a category C, C=3, the categories being entailment, contradiction, and neutrality, y c is a true label of the sample in the category c, y c is 1 if the true category of the sample is c, otherwise y c is 0.

[0015] The interview coaching method as described above, further comprising: taking a sum of the probability distribution values of the resume data and the interview answers in the entailment and neutral categories as the credibility score.

[0016] According to another aspect of the present application, an interview coaching device based on a large language model is provided, comprising: an acquisition module configured to acquire resume data of a job seeker and target job requirement data corresponding to the resume data; an interview question and answer generation module configured to input the resume data of the job seeker and the target job requirement data into a large language model and output a first interview question and a corresponding first interview answer; a relevance calculation module configured to input the first interview question and the first interview answer into a pre-trained relevance scoring model and output a relevance score, wherein the relevance scoring model is trained using a plurality of first sample data, the first sample data comprising an interview question and an interview answer, and the first sample data is divided into positive samples and negative samples according to the relevance of the interview question and the interview answer; a credibility calculation module configured to input the resume data of the job seeker and the first interview answer into a pre-trained credibility scoring model and output a credibility score, wherein the credibility scoring model is trained using a plurality of second sample data, the second sample data comprising resume data and interview answers, and the type of the second sample data is divided into: implication, contradiction and neutrality according to the logical relationship between the resume data and the interview answers; a comprehensive score calculation module configured to determine a comprehensive score according to the relevance score and the credibility score; and a question and answer optimization module configured to, if the comprehensive score is less than a preset threshold, input the relevance score, the credibility score and the first interview question into the large language model again, optimize the first interview question, output a second interview answer, and calculate a comprehensive score of the first interview question and the second interview answer, until the comprehensive score is greater than the preset threshold, and output the first interview question and the corresponding interview answer.

[0017] According to another aspect of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a set of computer program instructions, and the processor executes the set of computer program instructions on the memory to implement the interview coaching method based on a large language model as described above.

[0018] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a set of computer program instructions, and the set of computer program instructions is executed by a processor to implement the interview coaching method based on a large language model as described above.

[0019] The present application can effectively improve the quality of the final output answer by evaluating the relevance and credibility of the generated answers. It ensures that the answers provided by the system not only have semantic consistency, but also fit the actual needs in details and expressions, helping job seekers improve their interview performance. Through the optimization of relevance and credibility, the system can generate customized interview questions and answer suggestions for each job seeker, providing a highly personalized interview coaching experience. This customized coaching method helps job seekers improve their interview performance and enhance their competitiveness. BRIEF DESCRIPTION OF DRAWINGS

[0020] The preferred embodiments of the present application will be further described in details with reference to the drawings, wherein:

[0021] Figure 1 is a large language model-based interview tutoring method flowchart according to an embodiment of the present application.

[0022] Figure 2 is a training method flowchart of a relevance score model according to an embodiment of the present application.

[0023] Figure 3 is a training method flowchart of a relevance score model according to an embodiment of the present application.

[0024] Figure 4 is a large language model-based interview tutoring device structure schematic diagram according to an embodiment of the present application.

[0025] Figure 5 is a hardware structure schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0027] In the following detailed description, reference can be made to the various drawings that form a part of the present application and are used to illustrate specific embodiments of the present application. In the drawings, like reference numerals describe generally similar components throughout the several views. The various specific embodiments of the present application are described in sufficient detail to enable one of ordinary skill in the art and knowledge to understand the technical solutions of the present application. It should be understood that other embodiments or structural, logical or electrical changes to the embodiments of the present application can also be utilized.

[0028] Large Language Model (LLM) is a natural language processing (NLP) technology based on artificial intelligence, which is usually trained by deep learning method and can understand, generate and process human language. Large language models are usually trained using large amounts of text data to master the structure, grammar, word meaning and context relationship of language. The large language model in this application can learn complex language patterns, which can generate relevant interview questions according to the resume text of the job seeker, and customize the interview answers corresponding to the interview questions according to the information of the resume text.

[0029] Figure 1 is a flowchart of an interview coaching method based on a large language model according to an embodiment of the present application. As shown in Figure 1 , the method comprises:

[0030] S11, obtaining resume data of a job seeker and target job requirement data corresponding to the resume data;

[0031] S12, inputting the resume data of the job seeker and the target job requirement data into a large language model and outputting first interview questions and corresponding first interview answers;

[0032] S13, inputting the first interview questions and the first interview answers into a pre-trained relevance scoring model and outputting a relevance score, the relevance scoring model being trained using a plurality of first sample data, the first sample data comprising interview questions and interview answers, and the first sample data being divided into positive samples and negative samples according to the relevance of the interview questions and the interview answers;

[0033] S14, inputting the resume data of the job seeker and the first interview answers into a pre-trained credibility scoring model and outputting a credibility score, the credibility scoring model being trained using a plurality of second sample data, the second sample data comprising resume data and interview answers, and the type of the second sample data being divided into: implication, contradiction and neutrality according to the logical relationship between the resume data and the interview answers;

[0034] S15, determining a comprehensive score according to the relevance score and the credibility score;

[0035] S16, if the comprehensive score is less than a preset threshold, inputting the relevance score and the credibility score into the large language model again, outputting second interview answers, and calculating the comprehensive score of the first interview questions and the second interview answers until the comprehensive score is greater than the preset threshold, and outputting the first interview questions and the corresponding interview answers.

[0036] The resume data of the job seeker includes basic information, education background, work experience, project experience, and personal skills. The target job requirement data is the job that the job seeker wants to obtain, and the recruitment will retrieve the general job requirement text according to the job title. Before inputting the resume data and target job requirement data into the large language model, the text information needs to be preprocessed, and the preprocessing steps include:

[0037] Text cleaning: remove irrelevant characters, special symbols, HTML tags, spaces, and redundant line breaks in the resume and job requirement text to ensure the cleanliness and standardization of the text. This step can improve the accuracy of the model's understanding of the text content.

[0038] Tokenization and lemmatization: Tokenize the text to break down sentences into words or phrases. At the same time, restore the word form (such as restoring the different tenses of verbs to the original form, and restoring the plural form of nouns to the singular form) to reduce the redundancy and variants of the vocabulary that interfere with the model analysis.

[0039] Stop word filtering: remove common words without actual meaning (such as "of", "and", "is", etc.) to retain meaningful key content and ensure that the model focuses on important information such as education background, work experience, etc.

[0040] Entity recognition and extraction: through the named entity recognition (NER) technology in natural language processing, identify and extract important entity information in the resume, such as name, position, company name, time, and place. Similarly, perform entity recognition on the job information to extract job title, job responsibilities, and skill requirements.

[0041] Data standardization: standardize the format of dates and numerical information (such as work experience, education, etc.) in the resume and job data to make them consistent, so that the model can more accurately understand and compare different data points.

[0042] Vectorization processing: encode the resume and job information to convert the text into numerical form suitable for model processing, generating vector indexes for the resume and job. This step embeds the text information in a high-dimensional space, allowing the large language model to better understand the semantic relationship between texts.

[0043] De-duplication and content verification: check and remove duplicate content in the resume or job information to ensure that the data input into the model is concise and has no redundancy. At the same time, verify the completeness and accuracy of the resume and job information to avoid missing key data.

[0044] The processed resume data and target job requirement data are input into a large language model. The large language model will generate multiple interview questions based on the parsed resume information and job information according to different question types, including basic background questions, behavioral interview questions, and scenario simulation questions. Then, the most relevant content is queried from the resume and job vector index for each generated interview question to determine the best answer material. The queried content is used as context and submitted to the large model along with the interview question to generate a preliminary interview answer, which is the first interview answer.

[0045] The large language model of the present application includes but is not limited to any one of the following: Tong Yiqian, Wenxin Yanyan, and Zhipu glm. Alternatively, the large language model can also be fine-tuned in the vertical field on the open source version provided by the supplier of the large model Tong Yiqian, Wenxin Yanyan, and Zhipu glm.

[0046] The relevance of the present application solves the problem of determining whether the interview answer generated by the large model is relevant to the interview question. For example, the interview question is "What is your graduation school?", and the interview answer is "I have worked in XX company", and the relevance of the two is irrelevant, which is a negative sample, indicating that the interview answer is not a suitable answer to the interview question (it can be completely irrelevant or partially relevant). If the answer is "I graduated from XX University", the relevance of the two is relevant, which is a positive sample, indicating that the interview answer is a suitable answer to the interview question.

[0047] The credibility of the present application solves the problem of determining whether the interview answer generated by the large model is contradictory to the information in the job seeker's resume data. The inclusion relationship between the interview answer and the relevant content in the resume is investigated, and the consistency of the information is investigated. For example, part of the answer is "work for 5 years", and the resume information is "work for 3 years", the result is a contradictory relationship, and the generated answer may not be consistent with the resume content, and the credibility is low. If the resume information is "work for 3 years", the relationship is an inclusion relationship, indicating that the generated answer is semantically consistent with the resume content, and the credibility is high; if the resume information is "graduated for 5 years", the relationship is neutral, indicating that the generated answer has no direct contradiction or inclusion relationship with the resume content.

[0048] The scoring module of the present application can effectively improve the quality of the final output answer by evaluating the relevance and credibility of the generated answer. It ensures that the answers provided by the system not only have semantic consistency, but also meet the actual needs in terms of details and expression, helping job seekers improve their interview performance. Through the optimization of relevance and credibility evaluation, a customized interview question and answer suggestion can be generated for each job seeker, providing a highly personalized interview coaching experience. This customized coaching method helps job seekers improve their interview performance and enhance their competitiveness.

[0049] Further, compared with the traditional artificial interview coaching service, the technical solution greatly reduces the cost of interview coaching by automatically generating questions and answers. It can provide services to a large number of job seekers at the same time, and is not limited by the limited resources of experts, and has stronger expansibility and popularity.

[0050] According to an embodiment of the present application, the relevance score model is trained using the relevance scores of the same interview question with relevant interview answers and irrelevant interview answers. The first sample data of the present application is a triple, and the form of a triple is: (interview question, positive sample answer, negative sample answer list) triple. The positive sample answer is a suitable answer to the question, and the negative sample answer is an unsuitable answer (which can be completely irrelevant or partially relevant) to the same question. The positive sample answer and the negative sample answer of the same interview question are input into the relevance score model, and the relevance score model optimizes its parameters in a contrastive learning manner, effectively improving the training efficiency of the relevance score model. Thus, more accurate and actual job requirements meeting answers are provided to job seekers in interview coaching, improving the quality and effectiveness of interview coaching.

[0051] Figure 2 is a training method flowchart of the relevance score model according to an embodiment of the present application. As shown in Figure 2 , the method comprises:

[0052] S131, constructing a plurality of first sample data, wherein the positive sample comprises an interview question and an interview answer related to the interview question, and the negative sample comprises an interview question and an interview answer unrelated to the interview question;

[0053] S132, indexing the interview question and the interview answer in the first sample data to obtain a first sample sequence encoding, the first sample sequence encoding comprising a classification identifier for indicating whether the interview question and the interview answer are related;

[0054] S133, inputting the first sample sequence encoding into the relevance score model to output a relevance score;

[0055] S134, updating the model parameters in the relevance score model by minimizing the first loss function value until the first loss function value is less than the first loss threshold, the first loss function comprising the relevance score of the positive sample and the relevance score of the negative sample.

[0056] The application takes the interview question and the interview answer in the triple as a pair of input samples, inputs into the correlation score model to be trained, and the correlation score model can be a cross encoder model (for example, bge-reranker) which encodes the text. The cross encoder considers the entire sequence of inputs and encodes the semantic context of the question and the answer together, inputs the encoded input into the model, and the model outputs a correlation score (usually between 0 and 1) to measure the matching degree between the answer and the interview question. The score is used to evaluate whether the generated answer semantically answers the question.

[0057] The input of the cross encoder is a text pair of (interview question q, answer a), which needs to be preprocessed so that the cross encoder can accept the text and generate a corresponding vector representation. Specifically, first, the two sentences are spliced by a separator [SEP], and a special symbol [CLS] is added at the beginning of the sentence as a classification task marker, i.e., a classification identifier.

[0058] Then, the spliced text is segmented (sentence -> word sequence) and indexed (word sequence -> id sequence), and finally a sequence similar to the above format: [CLS], q0, q1, …, qn, [SEP], a0, a1, …, am can be obtained as the input of the encoder.

[0059] Finally, the spliced text sequence is input into the correlation score model. Since the correlation score model processes the interview question and the interview answer at the same time, the model can capture the mutual dependence and interaction information between the interview question and the answer. In this process, the model generates a context vector for each word in the input sequence. The output vector corresponding to the last layer of the “[CLS]” marker is extracted as the comprehensive representation vector of the entire spliced text. The vector is input into a linear layer, and a scalar score is output, which can be used as the semantic correlation score between the interview question and the answer. The calculation process is represented as s θ (q,a

[0060] score1=s θ (q,a + );

[0061] For a piece of training data, the loss function can be represented by the following formula:

[0062]

[0063] Where lossA is the first loss function value, s θ (q,a + ) is the correlation score of the positive sample, is the correlation score of the negative sample, and k is the number of negative samples.

[0064] In the contrastive learning training process, by minimizing the contrastive learning loss value of the cross-encoder, the model parameters of the cross-encoder are updated, so that the text similarity between the interview questions and the positive sample answers is as high as possible, and the text similarity between the interview questions and the negative sample answers is as low as possible.

[0065] The scheme constructs positive and negative sample datasets, and evaluates the semantic relevance of interview questions and interview answers by using a relevance scoring model. The model converts interview questions and answers into numerical vector representations based on a deep learning encoder, and performs supervised learning on positive and negative samples through a classification identifier. The model uses a loss function optimization strategy to maximize the score of relevant answers and minimize the score of irrelevant answers, thereby gradually improving the ability to judge the relevance of answers during the training process.

[0066] Figure 3 is a flowchart of a training method of a relevance scoring model according to an embodiment of the present application. As shown in Figure 3 , the method comprises:

[0067] S141, constructing a plurality of second sample data, the second sample data comprising resume data and interview answers;

[0068] S142, indexing the resume data and interview answers in the second sample data to obtain second sample sequence encoding, the second sample sequence encoding comprising a classification identifier for representing the logical relationship between the resume data and the interview answers;

[0069] S143, inputting the second sample sequence encoding into the credibility scoring model to output an unnormalized prediction value on each logical relationship and converting it into a probability distribution value on each logical relationship;

[0070] S144, updating the model parameters in the credibility scoring model by minimizing the second loss function value until the second loss function value is less than the second loss threshold, the second loss function comprising the true type of the second sample data and the probability distribution value on each logical relationship.

[0071] The format of the second sample data of the present application is: [resume text, generated interview answer] sample pair, and the logical relationship (implication, contradiction or neutrality) between them is labeled. At the same time, open source datasets such as OCNLI, CSNLI, STS-B-Chinese can be used to expand the dataset.

[0072] First, the [resume text, generated interview answer] sample pair needs to be preprocessed. The preprocessing steps are similar to the above calculation of relevance, which are tokenization and indexing of the sample pair to obtain the second sample sequence encoding.

[0073] Then, the second sample sequence is input into a credibility scoring model. The credibility scoring model can be a Natural Language Inference (NLI) model trained in a supervised learning manner using a pre-trained model as a base (for example, based on BERT and its derivative models). The representation vector corresponding to the [CLS] token output by the model is input into a linear classification layer to obtain an unnormalized prediction value (logits) on each logical relationship category (implication, contradiction, and neutrality), and the logits are converted into a probability distribution (p entailment ,p contradiction ,p neutral ) by a Softmax function. The unnormalized prediction value is converted into a probability distribution value by the following formula:

[0074]

[0075] where p c is the probability that the second sample data belongs to category C, C = 3, and the categories are implication, contradiction, and neutrality, and Z c is the unnormalized prediction value of the sample data in category C.

[0076] Finally, the NLI model is trained using a cross-entropy loss function. For a single sample, the second loss function is represented by the following formula:

[0077]

[0078] where lossB is the second loss function value, p c is the probability that the sample data belongs to category C, C = 3, and the categories are implication, contradiction, and neutrality, and y c is the true label of the sample in category c. If the true category of the sample is c, then y c is 1, otherwise y c is 0.

[0079] If the second loss function value is greater than or equal to the second loss threshold, the parameters in the model are updated, and the second loss function value is continuously calculated until the second loss function value is less than the second loss threshold, indicating that the model meets the target requirements, and the optimization training is stopped, and the parameter values in the model are fixed.

[0080] After the NLI model is inferred, the sum of the probability distribution values of the resume data and the interview answer in the implication and neutrality categories (i.e., the contradiction rate) is used as the credibility score. The following formula represents:

[0081] score2 = p entailment + p neutral ;

[0082] wherein score2 is the credibility score, p entailment is the probability distribution value under the "implication" category of the logical relationship, p neutral is the probability distribution value under the "neutral" category of the logical relationship.

[0083] The present application can ensure that the interview answers are consistent in semantics with the resume by judging the logical relationship between the interview answers and the resume, i.e., calculating the correlation score, and avoid self-contradictory situations. Using probability distribution (such as implication, contradiction and neutrality) provides a quantitative evaluation method, which facilitates quick identification of which answers need to be optimized and which are already well matched. According to the credibility score, the current interview answers can be optimized and modified. The advantages of the interviewee can be better demonstrated in the interview answers, and the performance in the interview can be improved.

[0084] According to one embodiment of the present application, the correlation score and the credibility score are weighted and summed to obtain a comprehensive score. The comprehensive score is represented by the following formula:

[0085] S = score1 x W A + score2 x W B ;

[0086] wherein S is the comprehensive score, score1 is the correlation score, W A is the weight of the correlation score, score2 is the credibility score, and W B is the weight of the credibility score.

[0087] wherein the value range of W A is 0.5-0.6, and the value range of W B is 0.4-0.5.

[0088] The above describes the implementation of the embodiments of the present application and the advantages brought by the embodiments through multiple embodiments. The present application combines specific examples to describe the specific processing process of the embodiments of the present application in detail.

[0089] S1: Assume that a job seeker submits a resume and also submits the recruitment requirements of the target position "data analyst".

[0090] S2: The system uses NLP technology to analyze the resume and the job requirements, extracts the key information, and uses a pre-trained encoder to segment and vectorize the corresponding text, constructs a vector index, so as to facilitate subsequent matching and scoring.

[0091] S3: Interview question generation

[0092] Based on the analyzed resume information and job requirements, the system generates multiple interview questions, taking the following question as an example:

[0093] Q1. Please briefly describe your work experience in the field of data analysis.

[0094] S4: Preliminary answer generation

[0095] Using Q1, the following information (abstract) is retrieved from the resume index and job requirement index:

[0096] - Resume context: "Company A (2019-present): Senior Data Analyst, job responsibilities:..., Company B (2016-2018): Junior Data Analyst, job responsibilities: …"

[0097] - Job requirement context: "Requires experience in data analysis and machine learning."

[0098] Based on the interview question, resume context, and job requirement context, a preliminary interview answer is generated:

[0099] A1: "I worked as a junior data analyst at Company A, mainly responsible for data cleaning and analysis. After joining Company B, I was promoted to senior data analyst and participated in a certain project, using machine learning techniques to support decision-making processes."

[0100] S5: Scoring module

[0101] S5.1 Relevance score

[0102] - Q1&A1 relevance score: 0.65 (related, but the answer is not comprehensive, the match between the answer and the question is moderate)

[0103] S5.2 Credibility score

[0104] - Resume context&A1 credibility score: 0.3 (there is obvious inconsistency between the answer and the resume)

[0105] S5.3 Comprehensive score calculation

[0106] - A1: 0.65 (relevance) × 0.6 (relevance weight) + 0.3 (credibility) × 0.4 (credibility weight) = 0.51

[0107] - Threshold judgment: assuming the preset threshold is 0.85

[0108] - A1: 0.51 < 0.85 → not passed

[0109] S6: Result feedback and optimization

[0110] Since the comprehensive score of A1 is lower than the threshold, the interview question, resume and job-related content, previous generated answer, and previous dimension score results are fed back to the large model, and the interview answer is required to be optimized again.

[0111] Generate answer A2 again:

[0112] "I worked as a junior data analyst at B Company from 2016 to 2018, mainly responsible for data cleaning and analysis using SQL and Python, optimizing data processing procedures and improving efficiency. In 2019, I joined A Company and was promoted to senior data analyst, participating in a certain project and supporting decision-making processes using machine learning technology."

[0113] The scoring module evaluates A2: relevance score 0.75 (the answer is still lacking in persuasiveness), credibility score 0.9 (very reliable), comprehensive score = 0.75 x 0.6 + 0.9 x 0.4 = 0.81 < 0.85 → not passed

[0114] Generate answer A3 again:

[0115] "From 2016 to 2018, I worked as a junior data analyst at B Company, mainly responsible for collecting and organizing data from different departments, and using SQL and Python for preliminary data cleaning and analysis. In addition, I automated report generation through scripting, reducing manual operation time by 30 hours / month and improving team efficiency.

[0116] After joining A Company in 2019, I was promoted to senior data analyst and began conducting more complex statistical analysis on large data sets. I participated in multiple cross-department projects, using machine learning techniques such as linear regression and random forest algorithms to analyze business data and provide data-driven decision support to the team."

[0117] The scoring module evaluates A2: relevance score 0.88 (answer on topic and content detailed), credibility score 0.9 (very reliable), comprehensive score = 0.88 x 0.6 + 0.9 x 0.4 = 0.89 > 0.85 → passed, the system returns the optimized answer A3.

[0118] From the above, the scoring module scores the interview answers in terms of relevance and credibility. When the score is not qualified, based on the relevance and credibility scores, the current interview answer is optimized to make the optimized interview answer on topic and detailed, with high credibility and reliable content.

[0119] The optimized interview answer can effectively guide job seekers in dealing with interviews, at least in the following aspects:

[0120] (1) Increased interview success rate. By analyzing the relevance of the resume and job description, job seekers can more accurately convey their experience and skills. This accuracy not only lies in the consistency of the content, but also in the depth and details of the answers, allowing the interviewer to clearly understand the professional capabilities of the job seeker, thereby increasing the likelihood of passing the interview;

[0121] (2) Strong relevance of interview answers. By analyzing the weaknesses and strengths of the job seeker, the optimization process can help the job seeker focus on the most relevant content to the job requirements. This relevance allows the job seeker to effectively demonstrate the match between their skills and the job requirements, enhancing the purpose and persuasiveness of the answers.

[0122] (3) Improved matching degree in interviews. In interviews, job seekers can demonstrate their understanding and adaptability to the job through precise answers, which not only reflects their serious attitude towards the job, but also shows how they combine their background with the company's needs. This adaptability allows job seekers to stand out among many candidates and impress the interviewer.

[0123] Experimental comparison before and after using the scoring module

[0124] Control group: The first interview question and the first interview answer generated from the large language model are directly pushed to the job seeker and the interviewer;

[0125] Experimental group: The first interview question and the first interview answer generated from the large language model are the same as the control group, and the scoring module is used to optimize the first interview answer in terms of relevance and credibility, outputting the second interview answer. The first interview question and the second interview answer are pushed to the job seeker and the interviewer.

[0126] 1. Recruit 50 job seekers, randomly and evenly distribute them to the control group and the experimental group (25 in each group).

[0127] 2. Recruit 50 interviewers, randomly and evenly distribute them to the control group and the experimental group (25 in each group).

[0128] 3. The two groups of job seekers and interviewers make evaluations based on the interview questions and answers in their respective groups.

[0129] 4. Collect job seekers' and interviewers' satisfaction with interview questions and answers (satisfaction is 1, dissatisfaction is 0).

[0130] 5. Compare the evaluations of the control group and the experimental group, and analyze whether the optimized answers significantly improve the job seekers' satisfaction.

[0131] The experimental results are shown in Table 1 below:

[0132] Table 1

[0133] Group Interviewer satisfaction score (0 / 1) Candidate satisfaction score (0 / 1) Control group 15 / 25 18 / 25 Experimental group 20 / 25 22 / 25

[0134] Experimental conclusion: According to the above data, the job seeker satisfaction of the experimental group is significantly higher than that of the control group, indicating that the optimized answers generated by the large language model have a positive impact on the job seeker satisfaction. The interviewee satisfaction of the experimental group is also significantly higher than that of the control group, indicating that the optimized interview answers are more popular in the evaluation of the interviewers. Therefore, pushing the optimized interview questions and answers to the job seekers is beneficial to improve the job success rate of the job seekers and improve the user satisfaction.

[0135] Corresponding to the method embodiment of the present application, the present application also provides a large language model-based interview coaching device, as shown in Figure 4 The large language model-based interview coaching device 100 comprises:

[0136] The acquisition module 110 is configured to acquire resume data of a job seeker and target job requirement data corresponding to the resume data in a resume database and a job requirement database, respectively.

[0137] The interview question and answer generation module 120 is configured to input the resume data of the job seeker and the target job requirement data into a large language model and output first interview questions and corresponding first interview answers.

[0138] The relevance calculation module 130 is configured to input the first interview questions and the first interview answers into a pre-trained relevance scoring model and output a relevance score, wherein the relevance scoring model is trained by using a plurality of first sample data, the first sample data comprises interview questions and interview answers, and the first sample data is divided into positive samples and negative samples according to the relevance of the interview questions and the interview answers.

[0139] The credibility calculation module 140 is configured to input the resume data of the job seeker and the first interview answers into a pre-trained credibility scoring model and output a credibility score, wherein the credibility scoring model is trained by using a plurality of second sample data, the second sample data comprises resume data and interview answers, and the type of the second sample data is divided into: implication, contradiction and neutrality according to the logical relationship between the resume data and the interview answers.

[0140] The scoring module 150 is configured to determine a comprehensive score according to the relevance score and the credibility score.

[0141] The question and answer optimization module 160 is configured to input the relevance score, the credibility score and the first interview questions into the large language model again if the comprehensive score is less than a preset threshold, optimize the first interview questions, output second interview answers, and calculate the comprehensive score of the first interview questions and the second interview answers until the comprehensive score is greater than the preset threshold, and output the first interview questions and the corresponding interview answers.

[0142] Figure 5 Fig. 1 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application, which can be implemented as a server or other various terminal devices, such as a desktop personal computer, a tablet computer, a laptop computer, a mobile phone, etc., and includes a processor 601 and a memory 602, which stores a set of program instructions, and the set of program instructions stored in the memory 602 is executed by the processor 601 to implement any one of the above-mentioned recommendation information generation methods.

[0143] Specifically, the processor 601 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured as one or more integrated circuits that implement an embodiment of the present application.

[0144] The memory 602 can include a mass storage for data or instructions. By way of example and not limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 602 can include removable or non-removable (or fixed) media. Where appropriate, the memory 602 can be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory 602 is non-volatile solid-state memory.

[0145] The memory can include read-only memory (ROM), random access memory (RAM), a magnetic disk storage medium, an optical storage medium, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to implement the recommendation information generation method provided by the present application.

[0146] In one example, the electronic device can further include a communication interface 603 and a bus 604. The processor 601, the memory 602, and the communication interface 603 are connected through the bus 604 and complete communication therebetween.

[0147] The communication interface 603 is mainly used to implement communication between various modules, devices, units, and / or equipment in the embodiments of the present application.

[0148] Bus 604 includes a hardware, software, or both that couples components of the online data traffic metering device to each other. As an example without limitation, bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 604 can include one or more buses. Although the present embodiment describes and shows a particular bus, the present application contemplates any suitable bus or interconnect.

[0149] The present application also provides a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement any of the foregoing embodiments of the method for interview coaching based on a large language model. The computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. The storage medium can be a transitory computer-readable storage medium or a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium can include, but is not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Embodiments corresponding to such storage devices include, for example, a magnetic hard disk, a floppy disk, a CD-ROM based on CD, DVD or Blu-ray technology, and a persistent solid-state memory such as flash memory, a solid-state drive, etc.

[0150] The present application also provides a computer program product comprising a set of computer program instructions which, when executed by a processor, implement any of the foregoing embodiments of the method for interview coaching based on a large language model. The computer program product includes, but is not limited to, an application installation package, an application plug-in, a mini-program that can run in some applications, and the like in the form of being published on a website or an application store.

[0151] It should be understood that the present application is not limited to the particular configurations and processes described above and shown in the drawings. Detailed descriptions of known methods have been omitted for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of steps, after understanding the spirit of the present application.

[0152] The above examples are only for illustrating the present application, and are not intended to limit the present application. Those skilled in the art can make various changes and modifications without departing from the scope of the present application. Therefore, all equivalent technical solutions should belong to the scope of the present application.

Claims

1. An interview coaching method based on a large language model, characterized in that: include: Obtaining the resume data of job seekers and target job demand data corresponding to the resume data; Input the applicant's resume data and target position requirement data into the large language model and output the first interview question and the corresponding first interview answer; Inputting a first interview question and a first interview answer into a pre-trained relevance scoring model and outputting a relevance score, wherein the relevance scoring model is trained using a plurality of first sample data, wherein the first sample data includes interview questions and interview answers, and the first sample data is divided into positive samples and negative samples according to the relevance between the interview questions and the interview answers; Inputting the resume data and first interview answers of the job applicant into a pre-trained credibility scoring model and outputting a credibility score, wherein the credibility scoring model is trained using a plurality of second sample data, wherein the second sample data includes the resume data and the interview answers, and classifying the second sample data into the following types: implication, contradiction, and neutrality based on a logical relationship between the resume data and the interview answers, wherein the sum of the probability distribution values ​​of the resume data and the interview answers under the implication and neutrality categories, respectively, is used as the credibility score; Determine the comprehensive score based on the relevance score and the credibility score; If the comprehensive score is less than the preset threshold, the relevance score, credibility score, first interview question, first interview answer, resume data and target position requirement data are input into the large language model again, the second interview answer is output, and the comprehensive score of the first interview question and the second interview answer is calculated until the comprehensive score is greater than the preset threshold, and the first interview question and the corresponding interview answer are output.

2. The interview coaching method according to claim 1, characterized in that: The relevance scoring model is trained using the relevance scores of the same interview question with relevant interview answers and irrelevant interview answers.

3. The interview coaching method according to claim 1, characterized in that: The training process of the relevance scoring model includes: Constructing a plurality of first sample data, wherein positive samples include interview questions and interview answers related to the interview questions, and negative samples include interview questions and interview answers unrelated to the interview questions; Indexing the interview questions and interview answers in the first sample data to obtain a first sample sequence code, wherein the first sample sequence code includes a classification identifier for indicating whether the interview questions and the interview answers are related; Inputting the first sample sequence code into the relevance scoring model to output a relevance score; Model parameters in the relevance scoring model are updated by minimizing a first loss function value until the first loss function value is less than a first loss threshold, wherein the first loss function includes a relevance score of a positive sample and a relevance score of a negative sample.

4. The interview coaching method according to claim 3, characterized in that: The first loss function is expressed using the following formula: ； Among them, lossA is the first loss function value, is the correlation score of the positive sample, is the correlation score of the negative sample, and k is the number of negative samples.

5. The interview coaching method according to claim 1, characterized in that: The training process of the credibility scoring model includes: Constructing a plurality of second sample data, wherein the second sample data includes resume data and interview answers; Indexing the resume data and the interview answers in the second sample data to obtain a second sample sequence code, wherein the second sample sequence code includes a classification identifier for indicating a logical relationship between the resume data and the interview answers; Input the second sample sequence encoding into the credibility scoring model, output the unnormalized prediction value on each logical relationship and convert it into a probability distribution value on each logical relationship; By minimizing a second loss function value, model parameters in the credibility scoring model are updated until the second loss function value is less than a second loss threshold, where the second loss function includes a true type of the second sample data and a probability distribution value on each logical relationship.

6. The interview coaching method according to claim 5, characterized in that: The second loss function is expressed using the following formula: ； Among them, lossB is the second loss function value, p c is the probability that the sample data belongs to category C, C=3, the categories are implication, contradiction and neutral, y c is the true label of the sample in category c. If the true category of the sample is c, then y c is 1, otherwise y c is 0.

7. An interview coaching device based on a large language model, characterized in that: include: The acquisition module is used to obtain the resume data of the job seeker and the target position requirement data corresponding to the resume data; An interview question and answer generation module, which is used to input the applicant's resume data and target position requirement data into the large language model and output the first interview question and the corresponding first interview answer; a relevance calculation module, configured to input the first interview question and the first interview answer into a pre-trained relevance scoring model and output a relevance score, wherein the relevance scoring model is trained using a plurality of first sample data, the first sample data including the interview question and the interview answer, and to divide the first sample data into positive samples and negative samples based on the relevance between the interview question and the interview answer; a credibility calculation module, configured to input the applicant's resume data and first interview answers into a pre-trained credibility scoring model and output a credibility score, wherein the credibility scoring model is trained using a plurality of second sample data, wherein the second sample data includes resume data and interview answers, and the second sample data is classified into the following types: implication, contradiction, and neutrality based on the logical relationship between the resume data and the interview answers, wherein the sum of the probability distribution values ​​of the resume data and the interview answers under the implication and neutrality categories, respectively, is used as the credibility score; A comprehensive score calculation module is used to determine the comprehensive score based on the relevance score and the credibility score; The question-and-answer optimization module is used to input the relevance score, credibility score, first interview question, first interview answer, resume data and target position requirement data into the large language model again if the comprehensive score is less than the preset threshold, optimize the first interview answer, output the second interview answer, and calculate the comprehensive score of the first interview question and the second interview answer until the comprehensive score is greater than the preset threshold, and output the first interview question and the corresponding interview answer.

8. An electronic device, characterized in that: It includes a processor and a memory, wherein a computer program instruction set is stored in the memory, and when the processor executes the computer program instruction set in the memory, the interview coaching method based on a large language model described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that in, The computer-readable storage medium stores a computer program instruction set, which, when executed by a processor, implements the interview coaching method based on a large language model described in any one of claims 1-6.

Images