Personalized face test question generation method and device based on large model and electronic equipment

Abstract

The invention provides a personalized interview test question generation method and device based on a large model and electronic equipment. Key information in an interview resume is extracted based on an NLP technology; constructing a first cue word based on the post key information and a preset surface test question generation demand; the first prompt word is used for indicating the large model to generate interview questions conforming to the post applied by the interviewer; inputting the first cue word into a pre-trained large model to generate an initial surface test question set; constructing a second cue word based on the initial surface test question set and the personal key information; the second cue word is used for indicating the large model to fuse the initial surface test question set with the personal key information to generate personalized surface test questions; and inputting the second cue word into the pre-trained large model to generate the final face test question, the face test question is generated through the large model, the problems of question leakage, update lag and the like of a traditional test question bank are avoided, and personalized information is fused, so that the generated face test question is more targeted and more personalized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and electronic device for generating personalized interview questions based on a large model. Background Art

[0002] Traditional interview question banks are usually pre-designed and collected fixed question sets, which are more general and less targeted. With the development of technology and changes in recruitment needs, their limitations are gradually emerging.

[0003] Traditional interview question banks are usually a general set of fixed questions. They cannot dynamically extract and adjust suitable interview questions according to different job requirements and characteristics, as well as the interviewee's experience and background, resulting in a lack of pertinence. In addition, the questions in traditional question banks are usually "classic questions" that are widely known and circulated publicly. It is very easy for interviewees to find answers through online searches, which can easily lead to question leaks. In addition, traditional interview question banks need to be manually maintained by a large number of professionals, which is costly, has a low update frequency, and outdated question content, and cannot keep up with the pace of industry development and technological frontiers. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device and electronic device for generating personalized interview questions based on a large model, so as to solve the problems of the traditional question bank, fixed questions, lack of flexibility and targeting.

[0005] In a first aspect, a method for generating personalized interview questions based on a large model is provided, the method comprising: Extract key information from interview resumes based on NLP technology; key information includes at least: job key information and personal key information; job key information includes at least: applied position and professional skills; personal key information includes at least name, work experience and project experience; A first prompt word is constructed based on the key information of the position and the preset interview question generation requirements; the first prompt word is used to instruct the large model to generate interview questions that meet the position applied for by the interviewee; the preset interview question generation requirements at least include the number of test questions, the difficulty level of the test questions, and the type of test questions; Input the first prompt word into the pre-trained large model to generate an initial interview question set; Constructing a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to fuse the initial interview question set with the personal key information to generate a personalized interview question; Input the second prompt word into the pre-trained large model to generate the final interview question.

[0006] Optionally, the key information extracted from the interview resume based on NLP technology includes: Preprocessing the interview resume, the preprocessing at least includes removing non-text content and formatting symbols in the interview resume to obtain a preprocessed resume text; Segment the pre-processed resume text to obtain at least a text segment of the applied position, a text segment of professional skills, a text segment of work experience, and a text segment of project experience; Search for keywords of the position applied for by the candidate in the text segment of the position application through keyword matching technology; Use predefined skill vocabulary to find professional skill keywords in professional skill text segments; Identify work unit keywords and position keywords in work experience text segments based on the pre-trained NER recognition model; Extract project name keywords and project data keywords based on project experience text segments; All the keywords obtained from the preprocessed resume text are constructed into key information in a structured format.

[0007] Optionally, the difficulty of the test questions may be determined by: The preprocessed resume text is input into a pretrained classifier for classification to obtain the test difficulty level of the preprocessed resume text.

[0008] Optionally, after generating the initial interview question set, the method further includes: Extract the semantic features of each interview question in the initial interview question set based on the pre-trained BERT model, and convert the extracted semantic features into vectors; Use cosine similarity to calculate the similarity between all vectors; Sort the calculated similarities in descending order; Eliminate interview questions whose similarity is greater than the preset similarity threshold to remove duplicate questions.

[0009] Optionally, the training process of the BERT model includes: Get training samples from the interview question bank; In each iteration, the training samples are masked; The masked training samples are input into the BERT model for training until the preset number of iterations is reached.

[0010] Optionally, at each iteration, masking the training samples includes: Randomly select a preset proportion of samples to be masked from the training samples; Among the samples to be masked, a part of the samples is replaced with the MASK mark, and the rest remains unchanged.

[0011] Optionally, replacing a portion of the samples with MASK tags includes: A multi-level mask is used to replace a part of the samples with MASK tags. The multi-level mask includes word-level mask, phrase-level mask and sentence-level mask. The word-level mask, phrase-level mask and sentence-level mask are combined in a preset combination method, and the preset combination method is one of a random combination, a fixed combination and a custom combination.

[0012] Optionally, after eliminating interview questions with similarities greater than a preset similarity threshold, the method further includes: Determine whether the number of questions after deduplication meets the preset question number threshold; If not, continue to generate interview questions through the pre-trained large model; If it is satisfied, the step of constructing the second prompt word is executed.

[0013] In a second aspect, a device for generating personalized interview questions based on a large model is provided, the device comprising: An extraction unit is used to extract key information from interview resumes based on NLP technology; the key information includes at least: job key information and personal key information; the job key information includes at least: applied position and professional skills; the personal key information includes at least name, work experience and project experience; A first construction unit is used to construct a first prompt word based on key information of the position and a preset interview question generation requirement; the first prompt word is used to instruct the large model to generate interview questions that meet the position applied for by the interviewee; the preset interview question generation requirement at least includes the number of test questions, the difficulty level of the test questions, and the type of test questions; A first generating unit, used for inputting a first prompt word into a pre-trained large model to generate an initial interview question set; A second construction unit is used to construct a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to merge the initial interview question set with the personal key information to generate a personalized interview question; The second generation unit is used to input the second prompt word into the pre-trained large model to generate a final interview question.

[0014] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement any method step described in the first aspect when executing a program stored in the memory.

[0015] According to a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0016] The present invention provides a method, device and electronic device for generating personalized interview questions based on a big model, which extract key information from an interview resume based on NLP technology; construct a first prompt word based on job key information and preset interview question generation requirements; the first prompt word is used to instruct the big model to generate interview questions that meet the job application of the interviewee; the first prompt word is input into the pre-trained big model to generate an initial interview question set; a second prompt word is constructed based on the initial interview question set and personal key information; the second prompt word is used to instruct the big model to fuse the initial interview question set with personal key information to generate personalized interview questions; the second prompt word is input into the pre-trained big model to generate the final interview question. Compared with the traditional question bank extracting fixed questions, the present invention constructs the first prompt word to instruct the big model to generate interview questions that are consistent with the job application of the interviewee and are suitable for the interview difficulty, and fuses the interviewee's personalized information to construct the second prompt word to instruct the big model to generate interview questions with the interviewee's personalized characteristics, thereby making the generated interview questions more targeted and personalized, and improving the interviewee's interview experience. In addition, since the interview questions are dynamically generated by a large model based on the information of each interview resume, it avoids the common problems in traditional question banks, such as question duplication, question bank leakage, and routine, making the test questions fresh and variable.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a method for generating personalized interview questions based on a large model provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of a large-model-based personalized interview question generation device provided by an embodiment of the present invention is shown; Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0021] Considering that traditional interview question banks are usually a general set of fixed questions, it is impossible to dynamically extract and adjust suitable interview questions according to different job requirements and characteristics, as well as the interviewee's experience and background, resulting in a lack of pertinence. In addition, the questions in traditional question banks are usually "classic questions" that are widely known and circulated publicly. It is very easy for interviewees to obtain answers through online searches, which easily leads to question leaks. In addition, traditional interview question banks need to be manually maintained by a large number of professionals, which is costly, has a low update frequency, and the question content is outdated, and cannot keep up with the pace of industry development and technological frontiers.

[0022] Based on this, an embodiment of the present invention provides a method and device for generating personalized interview questions based on a large model, which is described below through an embodiment.

[0023] The embodiment of the present invention provides a method for generating personalized interview questions based on a large model, such as Figure 1 As shown, the method comprises the following steps: Step S101: extract key information from the interview resume based on NLP (Natural Language Processing) technology.

[0024] In this step, the key information at least includes: job key information and personal key information; the job key information at least includes: applied position and professional skills; the personal key information at least includes name, work experience and project experience.

[0025] In one example, the professional skills that a computer candidate should understand or master are at least one of the following: programming language, algorithm, framework, platform, tool, data structure, etc.

[0026] Step S102: construct a first prompt word based on key position information and preset interview question generation requirements; the first prompt word is used to instruct the large model to generate interview questions that meet the interviewee's application for the position.

[0027] In this step, the preset interview question generation requirements include at least the number of questions to be generated, the difficulty level of the questions, the type of questions, etc.

[0028] The number of test questions can be, for example, 10 or 20 questions. The set number of test questions should ensure that it can meet the interview needs while leaving a sufficient question pool for subsequent screening.

[0029] In one example, the types of test questions include: single-choice questions (such as choosing one from four); multiple-choice questions (such as selecting all correct answers); and essay questions (such as what solutions are there to implement a certain function, and what are their advantages and disadvantages; how to design a system that supports high concurrency and high availability, etc.).

[0030] Among them, single-choice and multiple-choice questions usually require clear options and answers, while essay questions are more open. By pre-setting appropriate test questions, it is convenient to evaluate the candidate's technical level, thinking ability, comprehensive ability, innovation ability, and practical problem-solving ability.

[0031] In one example, the difficulty levels of the test questions include: easy, medium, and difficult. For example, for "deep learning", the easy question is "What is the role of the activation function in deep learning? Please list and briefly describe three common activation functions", while the difficult question is "Training a deep neural network, it is found that there is a serious imbalance of positive and negative sample categories in the training data set. If you are using the cross entropy loss function and the standard gradient descent algorithm. Please briefly describe the possible reasons for the poor performance of the model and propose ways to alleviate the problem." By constructing prompt words, large models can generate test questions of different depths and difficulties.

[0032] In a specific example, the first prompt word is as follows: Generate a set of interview questions for a candidate applying for a deep learning engineer position.

[0033] Key information of the position: Job Title: Deep Learning Engineer Main responsibilities: demand analysis and solution design, data processing and preparation, model development and training, model evaluation and optimization, model deployment and maintenance.

[0034] Professional skills: Have solid mathematical knowledge. Proficient in at least one programming language, such as Python. Familiar with mainstream deep learning frameworks and algorithms, such as TensorFlow and PyTorch. Master data processing and analysis skills. Have strong problem-solving and debugging skills, and be able to quickly locate and solve problems that arise during model training and deployment.

[0035] Preset interview question generation requirements: Total number of interview questions: 50; Difficulty level: Medium; Question type: Essay; Output format: json format.

[0036] The embodiment of the present invention can guide the large model to generate test questions that are highly relevant to the candidate's background and meet the job requirements through the content and structure of the first prompt word, so that the interview questions are closer to the actual work scenario and can more accurately and comprehensively evaluate the interviewee's true ability.

[0037] Step S103: input the first prompt word into the pre-trained large model to generate an initial interview question set.

[0038] In an embodiment of the present invention, the large model can adopt a large language model of the GPT series, which is pre-trained and fine-tuned based on the Transformer architecture. In order to solve the timeliness problem of the large model, a regular training method is used to fine-tune the large model, and the RAG (Retrieval-Augmented Generation) technology is combined to supplement the content in specific fields. RAG allows the large model to not only rely on the knowledge learned by the internal parameters when generating test questions, but also dynamically retrieve the most relevant information from the external database as a supplement. This means that even if the model itself does not have the latest knowledge, it can obtain and utilize the latest information through queries. Thereby increasing the accuracy of the generated test questions. By combining regular fine-tuning with RAG, the timeliness and accuracy of the large model are improved.

[0039] At the same time, in order to avoid overfitting and underfitting problems during fine-tuning of large models, some strategies can be adopted, such as regularization (weight decay, discarding), reducing the learning rate, early stopping (training too long), observing the learning curve, etc., to improve the comprehensive generalization ability of the model.

[0040] Step S104: construct a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to merge the initial interview question set with the personal key information to generate personalized interview questions.

[0041] In this step, the interview questions are combined with the interviewee's work experience, project experience and other personal key information again, and input into the big model to personalize the questions. Ultimately, the system generates a set of personalized questions that are tailored to each individual, and are finally stored in the question bank for interviews.

[0042] In one example, the second prompt word is as follows: Please use the personal key information and initial interview question set provided below to generate a set of personalized interview questions for the candidate.

[0043] Key personal information: Years of experience: [X] years; Skill set: Solid mathematical knowledge, including linear algebra, calculus, probability theory and mathematical statistics; proficient in Python programming language, familiar with various machine learning and deep learning algorithms and frameworks, such as linear regression, logistic regression, decision tree, neural network, TensorFlow, PyTorch, etc.; Work experience: Worked as [position name] at [Company X], responsible for [brief description of responsibilities]; Previously worked at [Company Y] as [position name], responsible for [brief description of responsibilities]; Project Experience: Participated in [Project A], which involved [Technology Stack / Tools / Methodology] and achieved [Project Outcome]; Participated in [Project B], which involved [Technology Stack / Tools / Methodology] and achieved [Project Outcome]; Initial interview question set: [General interview question 1] [General interview question 2] … Personalized rewriting requirements: Incorporate the candidate’s work experience, skills, and project experience into every interview question; Output format: json format.

[0044] Step S105: Input the second prompt word into the pre-trained large model to generate the final interview question.

[0045] In this step, fine-tune the pre-trained open source large model to adapt to the interview scenario. The fine-tuning process includes: Step 1: Data preparation Collect a large number of interview questions from different fields, candidate resumes and personal information, and personalized interview question examples.

[0046] Step 2: Model Architecture Design Choose an appropriate deep learning architecture according to task requirements. For example, the Transformer model is widely adopted due to its excellent performance in natural language processing tasks.

[0047] Embedding layer optimization: Optimize the embedding layer used to process text input so that the model can capture the relationship between lexical semantics and sentence structure.

[0048] The third step is to set the training objectives and model parameters The loss function is defined by calculating the similarity between the generated interview questions and the actual valid questions.

[0049] Set appropriate hyperparameters such as batch size, learning rate, number of iterations, etc.

[0050] Step 4: Fine-tune The collected data is input into the large model architecture for fine-tuning iterative training, and the changing trend of the loss function is observed until the number of iterative training times is reached.

[0051] In one example, the resulting personalized interview questions are as follows: (1) In your previous deep learning projects in the past X years, what optimization solutions and measures have you used? For example, what regularization methods have you used to address the overfitting problem in model training? In terms of hyperparameter tuning, what strategies have you adopted to improve model performance? (2) Based on your work experience at Company X and your experience in Project X, please explain the basic principles and architecture of deep learning models, especially the Transformer-based architecture or the application of convolutional neural networks (CNNs). Have you encountered problems such as gradient vanishing and gradient exploding in your projects? How did you solve these problems? The present invention constructs a first prompt word to indicate that the big model can generate interview questions that match the position applied for by the interviewee and are suitable for the interview difficulty, and integrates the interviewee's personalized information to construct a second prompt word to indicate that the big model can generate interview questions with the interviewee's personalized characteristics, thereby making the generated interview questions more targeted and personalized, and improving the interviewer's interview experience. In addition, since the interview questions are dynamically generated by the big model based on the information of each interview resume, the common problems of question duplication, question bank leakage, and routine in the traditional question bank are avoided, making the questions fresh and variable.

[0052] Based on the above embodiment, the key information extracted from the interview resume based on NLP technology includes: Step S101A: preprocessing the interview resume, where the preprocessing at least includes removing non-text content and formatting symbols in the interview resume to obtain a preprocessed resume text.

[0053] In this step, non-text content is removed by: Use regular expressions or other text parsing tools to remove non-text elements such as images and tables from PDF or Word documents.

[0054] If it is in HTML format, remove the tags through an HTML parsing library (such as BeautifulSoup) to keep only the plain text.

[0055] In this step, formatting characters are removed in the following way: Use regular expressions to remove extra whitespace characters, line breaks, tabs, and other invisible characters.

[0056] For special symbols (such as asterisk * and underscore _), decide whether to retain or convert them to ordinary characters based on actual needs.

[0057] Unify the encoding format, for example, convert all texts to UTF-8 encoding to ensure consistency in subsequent processing.

[0058] The above processing results in a continuous and clean text string, namely the preprocessed resume text.

[0059] Step S101B: segment the pre-processed resume text to obtain at least an application position text segment, a professional skills text segment, a work experience text segment, and a project experience text segment.

[0060] In a feasible implementation, segmentation is achieved by the following steps: 1. Define rules: Based on common resume templates, pre-set keywords or phrases as identifiers for each section. For example, "applying for a position" corresponds to the application text section; "professional skills" corresponds to the professional skills text section; "work experience" corresponds to the work experience text section; "project experience" corresponds to the project experience text section.

[0061] 2. Use NLP word segmentation and syntactic analysis techniques to identify and segment different types of text paragraphs. Or use a simple rule-based approach to start a new paragraph when encountering the keywords defined above.

[0062] 3. Check and optimize: Make sure the content of each paragraph is relatively independent and complete to avoid cross-contamination. For some information that is difficult to clearly classify, consider classifying it into the most relevant paragraph.

[0063] 4. Output result: multiple orderly arranged text segments corresponding to different information categories.

[0064] Step S101C: searching for keywords of the position applied for by the candidate in the position application text segment by using keyword matching technology.

[0065] In this step, common job titles in the industry are collected to form a vocabulary. The vocabulary should include synonyms, abbreviations, and other variations. Then, the application text segment is traversed to find words that match any item in the vocabulary. Fuzzy matching algorithms can be used to improve accuracy, allowing for a certain degree of spelling errors or variations.

[0066] Step S101D: Use a predefined skill vocabulary to search for professional skill keywords in the professional skill text segment.

[0067] In this step, similarly, a detailed skill vocabulary is compiled according to the characteristics of the industry, including but not limited to programming languages, tool software, professional knowledge fields, etc. Referring to step S101C, these skill items are searched in the professional skill text segment, thereby obtaining a series of professional skill tags.

[0068] Step S101E: Identify work unit keywords and position keywords in the work experience text segment based on the pre-trained NER recognition model.

[0069] In this step, the work experience text segment is fed into the NER model for prediction. The NER model will mark the entities in each sentence and indicate their categories such as organization, position, etc.

[0070] Step S101F: Extract project name keywords and project data keywords based on the project experience text segment.

[0071] In a feasible implementation, the method in step S101E may be referred to to extract project name keywords and project data keywords.

[0072] Step S101G: construct all keywords obtained from the preprocessed resume text into key information in a structured format.

[0073] Different interviewers and question designers have different background knowledge and work experience, and have different understandings and perceptions of the depth and breadth of the same question. Simple questions may also be new challenges, and complex questions may be easily solved. Inappropriate test questions cannot accurately and comprehensively evaluate the interviewees. Therefore, it is very demanding for question designers to reasonably design test questions and accurately divide the difficulty coefficients of test questions, which is crucial for companies and interviewers.

[0074] Therefore, based on the above embodiment, the method for determining the difficulty level of the test question includes: The preprocessed resume text is input into a pretrained classifier for classification to obtain the test difficulty level of the preprocessed resume text.

[0075] In an embodiment of the present invention, the classifier uses a trained machine learning model or a deep learning model. Such a model can learn the mapping relationship between text and its corresponding difficulty level through a large amount of historical data. Traditional machine learning models: such as support vector machines, naive Bayes, etc. Deep learning models: such as convolutional neural networks, recurrent neural networks, long short-term memory networks, etc.

[0076] The classifier will output the probability scores of one or more difficulty levels based on its internal learning results. If the output is the probability distribution of multiple difficulty levels, a certain rule (such as selecting the level with the highest probability) can be used to determine the final difficulty level of the test question.

[0077] In a traditional question bank, a large number of questions are obtained from Internet sites. There are many types and numbers of questions, and similar repeated content is inevitable. The embodiment of the present invention uses a large model to dynamically generate interview questions, which can avoid the repetition of questions to a certain extent, but it cannot guarantee complete avoidance. Therefore, it is necessary to deduplicate the generated initial interview questions to avoid repetition of questions.

[0078] The current deduplication technologies include the following: 1. Chinese patent CN111209734A discloses a method and system for removing duplicate questions based on a test question bank. The actual internal deduplication method is based on the Jaccard similarity algorithm, which is an indicator for measuring the similarity between two sets and is often used to calculate the similarity between texts or sets. Its core idea is to measure the similarity of two sets by calculating the size of the intersection and union of the keyword elements of the two sets. However, this method only focuses on whether there are elements in the set and how many common elements there are, without considering the frequency and number of elements, which affects the similarity results when frequency information is very important. In addition, this method does not pay attention to semantic relationships, which leads to low similarity calculation results in processing relatively long texts or tasks that require deep semantic analysis, such as "artificial intelligence", "machine learning", "deep learning", etc. Although these words are different, they are similar concepts, but they will be regarded as completely different elements in the Jaccard similarity calculation.

[0079] 2. Chinese patent CN116070599A discloses an intelligent question bank generation and auxiliary management system, which mentions that the SimHash algorithm is used to remove duplicate test questions. The core idea of ​​the algorithm is to generate a fixed-length binary hash value through feature weight calculation of the text, and judge the similarity through the Hamming distance of the hash value. This algorithm also has the problem of being unable to accurately capture the differences in test questions, especially when the changes between test questions involve changes in logic, synonyms, similar word replacements, changes in expression methods, and other semantic levels. It lacks sensitivity. For example, "Please calculate the area of ​​a circle with a radius of 5" and "Area calculation: a circle with a radius of 5, find its area", these two questions are exactly the same in semantics, but due to the different order of text expression, the SimHash algorithm generates different hash values, so the two questions are judged to be completely different, which leads to misjudgment.

[0080] 3. Chinese patent CN105824798A discloses a method for removing duplicate questions from a test question database based on the similarity of test question keywords. It uses the TF-IDF value as a vector and simply uses the inner product to calculate the similarity, without considering the effect of the vector length on the result. The result of the inner product is affected not only by the direction of the vector, but also by the length of the vector. If the directions of two vectors are very close, but the length of one vector is much larger than that of the other vector, the inner product value will be magnified and cannot accurately reflect the similarity of the two vectors. In addition, there is a lack of consideration of semantic relevance, and the contextual dependence of keyword weights is not considered. In the test question removal method based on keyword similarity, it is usually assumed that the weights of all keywords are independent. However, in actual situations, the semantics and importance of keywords often depend on the context. For example, the meaning of "function" in math problems and programming problems is completely different. The "function" in math emphasizes mathematical mapping relationships and expressions; the "function" in programming focuses more on functionality and executability.

[0081] 4. Chinese patents CN108984702A and CN108664630A respectively disclose a test question comparison method and system and a test question deduplication method and device. The core idea of ​​the deduplication method used is to use Levenshtein Distance, also known as the edit distance algorithm, which is an indicator to measure the difference between two strings. That is, the minimum number of editing operations required to change one string into another string is used to measure the similarity of two texts. This algorithm has the problem of ignoring context, ignoring grammatical structure, and failing to recognize semantics. For example, "artificial intelligence" and "AI", "computer" and "computer" have exactly the same semantics, but the edit distance is large.

[0082] 5. Chinese patent CN105373594A discloses a method and device for screening repeated questions in a question bank, in which multiple deduplication algorithms are used, namely Jaro distance algorithm, Dice coefficient, Lucene cosine theorem, and Boolean model. Jaro distance algorithm is an algorithm for measuring the similarity between two strings. It calculates the similarity by considering the character order, matching characters, and character swaps between the two strings. If the position distance of two characters in the two strings is less than or equal to a certain threshold, they are considered to match. If the matching character positions are inconsistent, they need to be swapped, and the similarity is further adjusted by calculating the number of swaps; Dice coefficient is similar to Jaccard, a set-based similarity measurement method used to measure the similarity between two sets (or strings), which is a variant of Jaccard coefficient; Lucene cosine theorem uses TF-IDF value as a vector to calculate the cosine similarity between two vectors.

[0083] None of these four traditional methods have the ability to understand semantics. Jaro distance and Dice coefficient are based only on character or word matching and ignore the semantic similarity of words. Lucene cosine similarity relies on word frequency vectors (TF-IDF). Although it takes into account some contextual information, it still relies on surface matching of words based on statistics and cannot understand deep semantic associations. The Boolean model completely ignores semantic information and only determines whether the query word is accurately matched.

[0084] Therefore, the embodiment of the present invention uses an improved embedding vector model combined with cosine similarity to perform deduplication.

[0085] Based on the above embodiment, after generating the initial interview question set, the method further includes: Step S106: Extract semantic features of each interview question in the initial interview question set based on the pre-trained BERT model, and convert the extracted semantic features into vectors.

[0086] In an embodiment of the present invention, the BERT (Bidirectional Encoder Representations from Transformers) model introduces masked language modeling and next sentence prediction, combines it with a self-attention mechanism, and performs unsupervised pre-training on a large amount of corpus to understand the contextual relationship of words in a sentence, thereby capturing deeper semantic information in the text.

[0087] During pre-training, the BERT model randomly selects some words in the input text sequence for masking and lets the model predict these masked words. This allows the model to learn to infer missing information based on the surrounding context. Specifically, BERT randomly selects 15% of the words in the input text sequence for masking, and then replaces 80% of the 15% of the words with a special MASK tag, while the remaining 10% remain unchanged, and the other 10% are randomly replaced with other words. This is done to allow the model to learn not only the contextual information of the masked words, but also the contextual information of the unmasked words, thereby improving the generalization ability of the model.

[0088] For example, given the sentence "I like machine learning and deep learning, and am very interested in the field of artificial intelligence", if "like" is selected for masking, it may be replaced with MASK, or remain unchanged, or replaced with other random words such as "passion", etc.

[0089] The above masking process has three disadvantages: First, the masking pattern and content of each sentence sample are determined once in the preprocessing stage and will not change afterwards, which means that some words will never have the opportunity to be masked, and thus the context information cannot be learned more fully; Second, 15% of the words are selected for masking, of which 80% are replaced with the [MASK] tag, 10% remain unchanged, and 10% are randomly replaced with other words. Although this 10% is randomly replaced with other words, the sample diversity is increased, but these irrelevant words also increase the noise, making the model unable to focus more on learning the real context information, so that it may learn the wrong context association; Third, the masking only uses word-level masks, and the model can only learn the context information at the vocabulary level, but cannot fully capture the context relationship at the phrase and sentence levels.

[0090] Therefore, in view of the above shortcomings, the training process of the BERT model is improved, and its training process includes: Get training samples from the interview question bank; In each iteration, the training samples are masked; The masked training samples are input into the BERT model for training until the preset number of iterations is reached.

[0091] By adopting real-time masking, different words are dynamically selected for masking in real time at each training iteration. This means that during the training process, each word has a chance to be masked, which helps the model learn the contextual information of the vocabulary more fully.

[0092] To address the second shortcoming, at each iteration, mask processing of training samples includes: Randomly select a preset proportion of samples to be masked from the training samples; Among the samples to be masked, a part of the samples is replaced with the MASK mark, and the rest remains unchanged.

[0093] The embodiment of the present invention does not use random replacement, and no longer replaces 10% of the selected words with random words, but replaces all the selected words with MASK tags. This can reduce the model from learning wrong context associations, because randomly replacing irrelevant words may introduce noise and cause the model to overfit.

[0094] To address the third shortcoming, based on the above embodiment, replacing a portion of samples with MASK marks includes: A multi-level mask is used to replace a part of the samples with MASK tags. The multi-level mask includes word-level mask, phrase-level mask and sentence-level mask. The word-level mask, phrase-level mask and sentence-level mask are combined in a preset combination method, and the preset combination method is one of a random combination, a fixed combination and a custom combination.

[0095] In one example, a custom combination method is used to set different combination ratios, such as word-level masking accounts for 50% of the total mask ratio, phrase-level masking accounts for 30%, and sentence-level masking accounts for 20%.

[0096] The present invention adopts multi-level masking, that is, on the basis of using word-level masking, phrase-level masking and sentence-level masking are added, and different levels of combinations are performed, which helps the model to learn longer context dependencies, improve the robustness and generalization ability of the model, and enable the model to perform better on large-scale data and complex tasks.

[0097] In a specific example: Interview question 1: "Difference between Python's for loop and while loop"; Interview question 2: "What are the characteristics of Python's loop statements?" After vectorization, Interview Question 1 is represented as:

[0098] Interview question 2 is expressed as:

[0099] The converted vector , The semantic features involved include: word frequency, word order, relationship, classification, part of speech, theme, field, positive and negative, syntactic structure, context, weight, etc. Each of these features is a dimension, and the feature values ​​of each dimension together constitute a high-dimensional vector.

[0100] Step S107: Calculate the similarity between all vectors using cosine similarity.

[0101] In one example, there are four test questions, which are converted into vectors A, B, C, and D respectively; in this step, the cosine similarity is used to calculate the similarities between AB, AC, AD, BC, BD, and CD respectively.

[0102] In this embodiment of the present invention, the cosine similarity calculation formula is: (1); in, , Respectively represent the vector representation of the two test questions; It is the vector dot product, which represents the sum of the multiplication of two vectors in the same dimension. It is calculated as follows: (2); in, and They are vectors and The dot product is the i-th component of , where n is the number of dimensions of the vector. The dot product reflects the similarity of two vectors in the same direction. The larger the value, the closer the directions of the two vectors are.

[0103] Continuing from the previous example:

[0104]

[0105] But in fact, the result of the dot product is not only affected by the direction of the vector, but also by the length of the vector. The larger the vector length, the larger the dot product value will be, even if the directions of the two vectors are not exactly the same. Therefore, the dot product cannot be used directly to simply reflect the similarity, and the modulus length is also needed. and It is the modulus of the vector, which indicates the size of the vector. The modulus is the sum of the squares of the components of the vector, and then the square root is used to get the final length of the vector. The purpose of the square root here is to restore the sum of squares to the original scale of the vector (i.e. the dimension size), thereby obtaining a non-negative real value.

[0106] The role of the modulus is to normalize the vector, that is, and The length of becomes 1, eliminating the influence of length, focusing only on the directional consistency of the two, ensuring that the similarity depends only on the direction of the vectors, rather than their absolute size, so that their semantic similarity can be judged more accurately, ensuring semantic fairness and accuracy. The calculation formula of the modulus length is as follows: (3); Continuing from the previous example:

[0107] calculate:

[0108] 16 Continuing from the previous example:

[0109] calculate:

[0110]

[0111] Finally, the cosine of the two vectors, cos(θ), is equal to the dot product of A and B divided by the product of the magnitudes of A and B.

[0112]

[0113] Final result: cosine similarity ≈ 0.994 The value range of cosine similarity is [-1, 1]; 1 means that the similarities of the two test questions are positively correlated, the angle is 0°, and the directions are consistent.

[0114] 0 means that the similarities between the two test questions are irrelevant, the angle is 90°, and the directions are perpendicular.

[0115] -1 means that the two test questions are negatively correlated, with an angle of 180° and opposite directions.

[0116] Between 0 and 1, the two questions have some similarity, but are not exactly the same, and have some overlap. The cosine similarity calculated here is 0.994, and the value of cosine similarity is close to 1, indicating that the cosine similarity of the two vectors is very high. The system will think that the two questions are too similar in semantics, which means that only one of "What is the difference between Python's for loop and while loop" and "What are the characteristics of Python's loop statement" will be retained.

[0117] In an example, if the questions are "Please explain how the for loop in Python works and what are its characteristics" and "Please list the common sorting algorithms in Python", the content of these two questions is completely different. One asks about the principle of the for loop and the other asks about the sorting algorithm. Although both are Python programming-related questions, they focus on different knowledge points, so their cosine similarity is close to 0, indicating that they are not related.

[0118] Furthermore, if the questions are "How to use Python's for loop to traverse a list" and "How to avoid using for loops to traverse a list in Python", although these two questions both involve Python list traversal, their core issues are opposite. The first question asks how to use for loops to traverse, and the second question asks how to avoid using for loops to traverse, the purpose of which is to examine how to reduce time complexity and whether there are other faster ways. Obviously, these two questions are completely opposite in semantics, and their cosine similarity is close to -1, indicating that there is a complete negative correlation between them.

[0119] Step S108: sort the calculated similarities in descending order.

[0120] Step S109: Eliminate interview questions whose similarity is greater than a preset similarity threshold to remove duplicate questions.

[0121] For example, the system sets a similarity threshold such as 0.9. If the cosine similarity of two test questions is higher than this threshold, the test questions are judged to be highly similar in content, and only one of them is retained, thereby avoiding redundant test questions from entering the test question bank, thereby improving the diversity and effectiveness of the questions.

[0122] Cosine similarity provides an efficient and intuitive way to screen and optimize the similarity detection of test questions. Its function and purpose is to compare the similarity of test questions and determine whether their contents are repeated or semantically similar.

[0123] Based on the above embodiment, after eliminating the interview questions whose similarity is greater than the preset similarity threshold, the method further includes: Determine whether the number of questions after deduplication meets the preset question number threshold; If not, continue to generate interview questions through the pre-trained large model; If it is satisfied, the step of constructing the second prompt word is executed.

[0124] In a feasible implementation, after the interview questions are generated, the generated interview questions are stored in a question bank, and the AI ​​interviewer extracts questions from the question bank and captures the questions into audio to ask questions to the interviewee.

[0125] Based on the same inventive concept, a device for generating personalized interview questions based on a large model is provided, such as Figure 2 As shown, the device comprises: Extraction unit 201, used to extract key information from the interview resume based on NLP technology; the key information at least includes: job key information and personal key information; the job key information at least includes: applied position and professional skills; the personal key information at least includes name, work experience and project experience; The first construction unit 202 is used to construct a first prompt word based on the key information of the position and the preset interview question generation requirements; the first prompt word is used to instruct the large model to generate an interview question that meets the interviewee's application position; A first generating unit 203, configured to input the first prompt word into the pre-trained large model to generate an initial interview question set; A second construction unit 204 is used to construct a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to merge the initial interview question set with the personal key information to generate a personalized interview question; The second generating unit 205 is used to input the second prompt word into the pre-trained large model to generate the final interview question.

[0126] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Figure 3As shown, it includes a processor 301 , a communication interface 302 , a memory 303 and a communication bus 304 , wherein the processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0127] Memory 303, used for storing computer programs; The processor 301 is used to implement the steps of the method for generating personalized interview questions based on a large model when executing the program stored in the memory 303.

[0128] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0129] The communication interface is used for communication between the above electronic device and other devices.

[0130] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0131] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0132] The computer program product for the method for generating personalized interview questions based on a large model provided in an embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0133] The device for generating personalized interview questions based on a large model provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. Technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0134] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0138] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0139] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for generating personalized interview questions based on a large model, characterized in that: The method comprises: Extract key information from interview resumes based on NLP technology; the key information includes at least: job key information and personal key information; the job key information includes at least: applied position and professional skills; the personal key information includes at least name, work experience and project experience; A first prompt word is constructed based on the key information of the position and the preset interview question generation requirements; the first prompt word is used to instruct the large model to generate interview questions that meet the position applied for by the interviewee; the preset interview question generation requirements at least include the number of test questions, the difficulty level of the test questions, and the type of test questions; Inputting the first prompt word into a pre-trained large model to generate an initial interview question set; Constructing a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to merge the initial interview question set with the personal key information to generate a personalized interview question; The second prompt word is input into the pre-trained large model to generate a final interview question.

2. The method according to claim 1, characterized in that The key information extracted from the interview resume based on NLP technology includes: Preprocessing the interview resume, wherein the preprocessing at least includes removing non-text content and formatting symbols in the interview resume to obtain a preprocessed resume text; Segmenting the pre-processed resume text to obtain at least a job application text segment, a professional skills text segment, a work experience text segment, and a project experience text segment; Searching for keywords of the position applied for by the candidate in the text segment of the position application by using keyword matching technology; Searching for professional skill keywords in the professional skill text segment using a predefined skill vocabulary; Identify work unit keywords and position keywords in work experience text segments based on the pre-trained NER recognition model; Extract project name keywords and project data keywords based on project experience text segments; All the keywords obtained from the preprocessed resume text are constructed into key information in a structured format.

3. The method according to claim 2, characterized in that The method for determining the difficulty level of the test questions includes: The pre-processed resume text is input into a pre-trained classifier for classification to obtain the test difficulty level of the pre-processed resume text.

4. The method according to claim 1, characterized in that: After generating the initial interview question set, the method further includes: Extracting semantic features of each interview question in the initial interview question set based on the pre-trained BERT model, and converting the extracted semantic features into vectors; Use cosine similarity to calculate the similarity between all vectors; Sort the calculated similarities in descending order; Eliminate interview questions whose similarity is greater than the preset similarity threshold to remove duplicate questions.

5. The method according to claim 4, characterized in that The training process of the BERT model includes: Get training samples from the interview question bank; In each iteration, mask processing is performed on the training sample; The masked training samples are input into the BERT model for training until the preset number of iterations is reached.

6. The method according to claim 5, characterized in that In each iteration, mask processing is performed on the training sample, including: Randomly select a preset proportion of samples to be masked from the training samples; Among the samples to be masked, a portion of the samples are replaced with MASK marks, and the remaining portions remain unchanged.

7. The method according to claim 6, characterized in that Replacing a portion of the samples with MASK tags includes: A multi-level mask is used to replace a part of the samples with MASK tags. The multi-level mask includes a word-level mask, a phrase-level mask and a sentence-level mask. The word-level mask, the phrase-level mask and the sentence-level mask are combined in a preset combination method, and the preset combination method is one of a random combination, a fixed combination and a custom combination.

8. The method according to claim 4, characterized in that After eliminating the interview questions whose similarity is greater than a preset similarity threshold, the method further includes: Determine whether the number of questions after deduplication meets the preset question number threshold; If not, continue to generate interview questions through the pre-trained large model; If it is satisfied, the step of constructing the second prompt word is executed.

9. A device for generating personalized interview questions based on a large model, characterized in that: The device comprises: An extraction unit is used to extract key information from an interview resume based on NLP technology; the key information includes at least: job key information and personal key information; the job key information includes at least: applied position and professional skills; the personal key information includes at least name, work experience and project experience; A first construction unit is used to construct a first prompt word based on the key information of the position and the preset interview question generation requirements; the first prompt word is used to instruct the large model to generate interview questions that meet the position applied for by the interviewee; the preset interview question generation requirements at least include the number of test questions, the difficulty level of the test questions, and the type of test questions; A first generating unit, configured to input the first prompt word into a pre-trained large model to generate an initial interview question set; A second construction unit is used to construct a second prompt word based on the initial interview question set and the personal key information; the second prompt word is used to instruct the large model to merge the initial interview question set with the personal key information to generate a personalized interview question; The second generating unit is used to input the second prompt word into the pre-trained large model to generate a final interview question.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

Images