An interview system construction method based on a large language model multi-agent mechanism

Abstract

The application discloses an interview system construction method based on a large language model multi-agent mechanism and belongs to the field of artificial intelligence dialogue systems.The application implements the method as follows: a seed data set of an interview field prompt-question pair is generated;the seed data set based on ChatGPT is cyclically enhanced to form an updated seed data set;and a training data set is obtained after iteration for n times.The interviewer model is trained so that the interviewer model has an interview question generation capability.A role agent that determines an interview question order is designed.An environment agent that generates a text generation standard is designed.A memory agent that stores questions and answers and makes decisions is designed.The designed interview system is tested.The interview records obtained by the interview system are compared and evaluated to obtain an evaluation result.The interview system uses a large language model and a multi-agent mechanism to realize smooth and natural generation of interview questions, and through the answers of interviewees, interactive generation of more natural and logical interview questions is realized.

Description

TECHNICAL FIELD

[0001] The application relates to an interview system construction method based on a large language model multi-agent mechanism, and belongs to the field of artificial intelligence dialogue systems. BACKGROUND

[0002] In the current recruitment process, the interview link is often time-consuming and costly. The low efficiency of the interview is mainly reflected in the screening process of the candidate. The HR needs to spend a lot of time reading the resume, screening the suitable candidate, and arranging the interview. In addition, the questions in the interview process often lack pertinence and consistency, which leads to the inability to comprehensively evaluate the candidate's ability. At the same time, the subjectivity of the interview may cause excellent candidates to be overlooked, and not suitable candidates to be selected. These factors not only increase the operating cost of the enterprise, but also affect the quality and efficiency of the recruitment.

[0003] In order to solve these problems, the patent designs a large language model interview system based on a multi-agent mechanism. The system can automatically generate high-quality interview questions to ensure that each candidate receives the same standard evaluation. Through the precise question generation logic, the system can propose targeted questions according to the characteristics of different positions and candidates, so as to more effectively evaluate the skills and potential of the candidates. In addition, the system can also automatically record and analyze the interview process to provide data support for the HR, helping them make more objective and accurate recruitment decisions. SUMMARY

[0004] In view of the problem that the human cost in the interview field is not enough, the purpose of the application is to provide an interview system construction method based on a large language model multi-agent mechanism, to construct an interview data set of the interview system based on ChatGPT, to use a large language model and a multi-agent mechanism to realize the smooth and natural generation of interview questions, and to interactively generate the next interview question through the answers of the interviewers. The interview system constructed by the application is more natural and logical.

[0005] The purpose of the application is realized through the following technical solutions.

[0006] The application discloses an interview system construction method based on a large language model multi-agent mechanism, which comprises the following steps:

[0007] Step 1: Training of the interviewer model.

[0008] Step 1.1: Generating an interview field prompt-question pair seed data set.

[0009] According to the interview module, relevant data is investigated to generate a seed data set of the module. The data set is in the form of a prompt-interview question pair for the subsequent training of the large language model.

[0010] Step 1.2: Cycle enhancement based on the seed dataset of ChatGPT, form an updated seed dataset; after n iterations, get the training dataset, divide it into training set and test set.

[0011] Input the seed dataset of step 1.1 into ChatGPT to generate new data samples;

[0012] De-duplicate the generated new data samples to ensure the uniqueness of the new data;

[0013] Merge the de-duplicated data samples into the seed dataset of step 1.1 to form an updated seed dataset; after n iterations, get the training dataset, divide it into training set and test set;

[0014] Step 1.3: Use the training dataset obtained in step 1.2 to train the interviewer model, so that the interviewer model has the ability to generate interview questions.

[0015] Input the training set obtained in step 1.2 into the large language model for training; during the training process, control the training speed and effect according to the batch size and iteration number, and train the interviewer model; save the interviewer model parameters and optimizer state for subsequent testing and application;

[0016] Step 1.4: Test the interview question generation ability of the interviewer model constructed in step 1.3.

[0017] Input the test set data into the interviewer model of step 1.3 to get the corresponding question generation result; at the same time, calculate the text generation quality to evaluate the performance of the interviewer model; select the interviewer model with text generation quality meeting the preset requirements as the interviewer model passing the test.

[0018] Step two: design multi-agent in the interview system.

[0019] Step 2.1: Design role agent that determines the order of interview questions.

[0020] The interview system is divided into one-level dimension and two-level dimension, the one-level dimension is the module that needs to be interviewed at present, and the two-level dimension is the emphasis of each module; the role agent determines the order of one-level dimension and two-level dimension in the form of generating random numbers to ensure the randomness of the questions in the simulated real interview.

[0021] Step 2.2: Design environment agent that generates text generation standards.

[0022] The environment agent accepts the current environment, that is, the order of one-level dimension and two-level dimension output by the role agent as input prompt, and outputs the standard of ending the current environment question, that is, the standard of sufficient information collection;

[0023] Step 2.3: Design a memory agent that stores questions and answers and makes decisions.

[0024] The interviewer model obtained in step one is used to conduct interviews with interviewees. The memory agent stores all process records of the current interview. Based on the current environment and the standards generated by the environment agent, it is determined whether the interviewee's answer contains sufficient information. If so, the system jumps to the next environment to ask questions based on the output of the role agent. If not, the system continues to ask questions based on the questions in the current environment until enough information is collected, thus realizing the construction of the interview system.

[0025] It also includes step 2.4: testing the designed interview system.

[0026] Invite relevant personnel to use the interviewer model to experience the interview process. Based on their feedback, address any shortcomings and improve the overall user experience of the interview system.

[0027] It also includes step three, which involves comparing and evaluating the interview records obtained by the interview system to obtain evaluation results.

[0028] Step 3.1: Develop evaluation criteria for comparative evaluation.

[0029] To evaluate the interview records obtained using the interview system, the effectiveness of the system is determined by judging the consistency of the job suitability of the interviewees reflected in the interview records.

[0030] The evaluation criteria are: (1) Professional matching: whether the interviewee's professional skills match the job requirements;

[0031] (2) Educational background matching: Whether the interviewee's educational background matches the job requirements.

[0032] Step 3.2: Invite company HR and use the large model to score.

[0033] We obtained 100 interview process records and invited corporate HR personnel to score them. At the same time, we used a large model to score the interview process records and recorded two sets of scores.

[0034] Step 3.3: Compare and calculate the internal consistency and correlation between the large model and the scoring of the company's HR.

[0035] The Cronbach's Alpha coefficient and McDonald's ω coefficient were calculated for the two sets of scores. Both the Cronbach's Alpha coefficient and McDonald's ω coefficient were greater than 0.8, indicating that the data is reliable and that the internal consistency meets the preset requirements.

[0036] Pearson correlation coefficient and Spearman rank correlation coefficient were calculated for the two groups of scores, both Pearson correlation coefficient and Spearman rank correlation coefficient were greater than 0.8, indicating high correlation.

[0037] Step four: apply the interview system to enterprise recruitment, career development services, online education platforms, government talent selection, higher education institutions, customer service industries, interactively generate interview questions, and improve the service quality and efficiency of the above processes.

[0038] In enterprise recruitment, the interview system is integrated into the enterprise recruitment process, automates the preliminary screening process, quickly assesses the communication skills and professional skills of candidates through simulated interviews, reduces the workload of the human resources department, and improves the efficiency and quality of the entire interview process.

[0039] In career development services, the interview system serves as a personalized interview training tool for job seekers, providing real-time feedback and improvement suggestions to help job seekers quickly and accurately correct pronunciation errors and improve pronunciation levels, thereby increasing the chances of obtaining a position.

[0040] In online education platforms, the interview system is part of the students' career development, helping students adapt to the pressure of real interviews in advance by simulating interview environments, providing real-time feedback, and enhancing students' interview skills.

[0041] In the government talent selection process, the interview system serves as a fair and transparent auxiliary tool to help evaluate the comprehensive abilities of candidates and ensure the fairness of the selection process.

[0042] In higher education institutions, university career service centers use the interview system to help graduating students prepare for interviews, improve their employment rates, and enhance their career preparation levels.

[0043] In the customer service industry, the interview system is used to screen and train customer service representatives with excellent communication skills, thereby improving customer satisfaction and the quality of enterprise services.

[0044] Beneficial effects:

[0045] 1、The previous interview system often directly matches the interview questions through the interviewee's resume, without judging the next question through the interviewee's answers in the interview process; the previous interview system often generates questions based on rule matching and other methods, resulting in unnatural generated text; the previous interview system constructed by using a large model lacks certain logicality, which can make the system stability low and unable to completely collect all the information required for an interview, the interview system construction method based on the large language model multi-agent mechanism disclosed in the application constructs an interview dataset based on ChatGPT, generates fluent and rich text based on the large language model, and judges the generation of the next question based on the multi-agent mechanism through the interviewee's answers, so that the interview system is more natural and has logicality.

[0046] 2、The interview system construction method based on the large language model multi-agent mechanism disclosed in the application, the seed dataset based on ChatGPT is cyclically enhanced to form an updated seed dataset; after iteration n times, a training dataset is obtained; the interviewer model is trained to make the interviewer model have the interview question generation capability; a role agent that decides the interview question order is designed; an environment agent that generates a text generation standard is designed; a memory agent that stores questions and answers and makes decisions is designed; the designed interview system is tested; the interview records obtained by the interview system are compared and evaluated; the interview system uses the large language model and the multi-agent mechanism to realize the smooth and natural generation of interview questions, and through the answers of the interviewee, the interactive generation of more natural and logical interview questions is realized.

[0047] 3、The interview system construction method based on the large language model multi-agent mechanism disclosed in the application, the interview system is divided into a first dimension and a second dimension, the first dimension is a module currently requiring an interview, and the second dimension is the emphasis of each module; the role agent decides the order of the first dimension and the second dimension in a random number generation manner to ensure the randomness of the occurrence of questions in a real interview.

[0048] 4、The interview system construction method based on the large language model multi-agent mechanism disclosed in the application is applied to the fields of enterprise recruitment, career development services, online education platforms, government talent selection, higher education institutions and customer service industries, and interactive interview questions are generated, which has the following advantages.

[0049] ①Improve the interview performance of candidates: using the interview system can help candidates practice and improve their communication skills in simulated interviews. The real-time feedback provided by the system can help candidates better master language expression and non-verbal communication in interviews, improve their self-confidence and interview performance.

[0050] ②Optimize the recruitment process of enterprises: the interview system can serve as an efficient screening tool to quickly identify the communication skills and professional skills of candidates, thereby improving the quality and efficiency of the recruitment process.

[0051] ③Promote education and vocational training: educational institutions and vocational training centers can use the interview system as a teaching tool to help students and job seekers prepare for interviews, provide personalized feedback and suggestions, and promote educational equity and career development.

[0052] ④Increase employment opportunities: through the practice of the interview system, candidates can improve their interview skills and enhance their performance in real interviews, thereby increasing their chances of obtaining a position.

[0053] ⑤Reduce enterprise costs: automated interview systems reduce the human cost of human resources departments in the preliminary screening process, allowing enterprises to reallocate resources to more critical recruitment stages.

[0054] ⑥Improve user experience: the interview system provides a smooth and natural interactive experience, with questions generated by large language models and multi-agent mechanisms that are more close to real interview scenarios, allowing candidates to have a more realistic experience in practice.

[0055] ⑦Enhance the ability of candidates to self-assess: through the feedback of the system, candidates can more clearly recognize their strengths and weaknesses in interviews, thereby conducting targeted self-improvement.

[0056] ⑧Promote talent development and retention: enterprises can more accurately assess the potential and suitability of candidates through the system, which helps enterprises attract and retain key talents.

[0057] ⑨Support continuous learning and improvement: the interview system can serve as a continuous learning resource to help candidates continuously learn and develop to adapt to the changing job market needs.

[0058] ⑩Enhance cross-cultural communication: in a globalized work environment, the interview system supports candidates with multiple languages and cultural backgrounds, helping enterprises attract and select talents with international perspectives, enhancing the diversity and cross-cultural communication ability of the team. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 Flowchart of the interview system based on large language model constructed by the present application;

[0060] Figure 2 Flowchart of comparative evaluation of the interview system. DETAILED DESCRIPTION

[0061] In order to better illustrate the purposes and advantages of the present application, the content of the application is described below in conjunction with the drawings and examples

[0062] Embodiment 1

[0063] As shown in the figure, the embodiment discloses a method for constructing an interview system based on a large language model multi-agent mechanism, and the specific implementation steps are as follows: Figure 1 Step one: training of the interviewer model

[0064] Step 1.1: generating an interview field prompt-question pair seed dataset

[0065] Taking the recruitment in the field of artificial intelligence as an example, the interviewer module is designed in this example as: an education background interviewer, a scientific research experience interviewer, and a personality test interviewer, and three module seed datasets are generated respectively, in the form of prompt-interview questions, for subsequent training of the large language model; an example of the seed dataset is as follows:

[0066] Table 3.1 Seed dataset example

[0067]

[0068]

[0069] Step 1.2: cyclic enhancement of the seed dataset based on ChatGPT

[0070] First, input the seed dataset of step 1.1 into ChatGPT to generate new data samples;

[0071] Second, the generated new data samples are de-duplicated to ensure the uniqueness of the new data;

[0072] Finally, the de-duplicated data samples are merged into the seed dataset of step 1.1 to form an updated seed dataset; after iterating n times, a training dataset is obtained, which is divided into a training set and a test set; the final size of the training dataset is as follows:

[0073] Table 3.2 Data generation results

[0074]

[0075] Step 1.3: using the training dataset obtained in step 1.2 to train the three interviewer models to have the ability to generate interview questions;

[0076] Input the training set obtained in step 1.2 into the large language model for training; during the training process, the batch size and the number of iterations are used to control the training speed and effect, and after the training is completed, the interviewer model is obtained; save the interviewer model parameters and the optimizer state for subsequent testing and application;

[0077] Step 1.4: testing the interview question generation ability of the interviewer model constructed in step 1.3​

[0078] The test set data is input into the interviewer model of step 1.3, and the corresponding question generation result is obtained; at the same time, the text generation quality is calculated to evaluate the performance of the model; select the text generation quality, and get the interviewer model; the text generation quality test results of three interviewers are as follows:

[0079]

[0080] Table 3.6 personality test module experimental results

[0081]

[0082] Step two: design multi-agent in the interview system

[0083] Step 2.1: design role agent to determine the order of interview questions

[0084] The interview system is divided into primary and secondary dimensions. The primary dimension is the module currently needing to be interviewed, while the secondary dimension is the focus of each module. The role agent determines the order of the primary and secondary dimensions in the form of generating random numbers to ensure the randomness of the questions in the simulated real interview;

[0085] Step 2.2: design environment agent to generate text generation standards

[0086] The environment agent accepts the current environment, i.e. the order of the primary and secondary dimensions output by the role agent as input prompts, and outputs the end of the current environment question, i.e. the standard of sufficient information collection. An example of the information point standard of the environment agent is as follows:

[0087] Step 2.3: design memory agent to store questions and make decisions

[0088] Use the interviewer model obtained in step one to conduct an interview with the interviewee. The memory agent stores all the process records of the current interview. According to the current environment and the standard generated by the environment agent, it judges whether the interviewee's answer contains sufficient information. If yes, it jumps to the next environment according to the output of the role agent to ask questions. If not, it continues to ask questions according to the questions in the current environment until enough information is collected.

[0089] Table 3.3 example of information point standard generated by environment agent

[0090]

[0091] Step 2.4: test the designed interview system

[0092] Invite relevant personnel to use the model for interview experience, improve relevant deficiencies according to personnel feedback, and improve system use experience.

[0093] Step three, interview system comparison and evaluation method, process reference Figure 2 :

[0094] Step 3.1: Develop evaluation criteria for comparison and evaluation

[0095] Evaluate the interview records obtained by using the interview system, judge the consistency of the job matching degree of the interviewee reflected by the interview records, and judge the effectiveness of the system;

[0096] The evaluation criteria are: (1) Professional matching degree: whether the professional skills of the interviewee match the job requirements;

[0097] (2) Education experience matching degree: whether the education experience of the interviewee matches the job requirements;

[0098] Step 3.2: Invite enterprise HR and use large model scoring

[0099] Obtain 100 interview process records, first invite enterprise HR to score the interview process records; At the same time, use the large model to score the interview process records, record two scores.

[0100] Step 3.3: Compare and calculate the internal consistency and correlation of the scores of the large model and enterprise HR

[0101] Cronbach's Alpha coefficient and McDonald's omega coefficient are calculated for the two groups of scores, Cronbach's Alpha coefficient and McDonald's omega coefficient are greater than 0.8, the data is reliable, and the internal consistency meets the requirements; The experimental results are as follows:

[0102] Table 5.5 Comparison and evaluation of reliability coefficient calculation results

[0103]

[0104] Pearson correlation coefficient and Spearman rank correlation coefficient are calculated for the two groups of scores, Pearson correlation coefficient and Spearman rank correlation coefficient are greater than 0.8, and the correlation is high; The experimental results are as follows:

[0105]

[0106] Through training interviewers and developing multi-agent mechanism, a real-time interactive interview system is constructed, which can be evaluated by comparison and evaluation, thereby alleviating the difficulties of traditional interviews and saving human resources.

[0107] The above detailed description of the specific description, the purpose, technical scheme and beneficial effects of the application are further described in detail, it should be understood that the above description is only a specific embodiment of the application, and is not used to limit the protection scope of the application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. A method for constructing an interview system based on a large language model multi-agent mechanism, characterized in that: Comprising the following steps, Step one: training of the interviewer model; Step 1.1: generating an interview field prompt-question pair seed dataset; According to the interview module, research related materials to generate a seed dataset for this module. The dataset is in the form of prompt-interview questions, which will be used for subsequent training of large language models; Step 1.2: cyclic enhancement of the seed dataset based on ChatGPT to form an updated seed dataset; after n iterations, obtain the training dataset, which is divided into training set and test set; Input the seed dataset of step 1.1 into ChatGPT to generate new data samples; De-duplicate the generated new data samples to ensure the uniqueness of the new data; Merge the de-duplicated data samples into the seed dataset of step 1.1 to form an updated seed dataset; after n iterations, obtain the training dataset, which is divided into training set and test set; Step 1.3: use the training dataset obtained in step 1.2 to train the interviewer model to make the interviewer model have the ability to generate interview questions; Input the training set obtained in step 1.2 into the large language model for training; During training, control the training speed and effect according to the batch size and iteration number to train the interviewer model; save the interviewer model parameters and optimizer state for subsequent testing and application; Step 1.4: test the interview question generation ability of the interviewer model constructed in step 1.3; Input the test set data into the interviewer model of step 1.3 to obtain the corresponding question generation result; at the same time, calculate the text generation quality to evaluate the performance of the interviewer model; Select the interviewer model with text generation quality meeting the preset requirements as the interviewer model passing the test; Step two: design multi-agent in the interview system; Step 2.1: design role agent that decides the order of interview questions; The interview system is divided into first-level dimension and second-level dimension. The first-level dimension is the module currently needing to be interviewed, and the second-level dimension is the focus of each module. The role agent decides the order of the first-level dimension and the second-level dimension in the form of generating random numbers to ensure the randomness of the questions in the simulated real interview; Step 2.2: design environment agent that generates text sending standards; The environment agent accepts the current environment, i.e., the order of the first-level dimension and the second-level dimension output by the role agent as the input prompt, and outputs the standard for ending the current environment question, i.e., the standard for sufficient information collection; Step 2.3: design memory agent that stores questions and answers and makes decisions; Use the interviewer model obtained in step one to interview the interviewee. The memory agent stores all process records of the current interview. According to the current environment and the standard generated by the environment agent, judge whether the interviewee's answer contains sufficient information. If yes, jump to the next environment for questioning according to the output of the role agent. If not, continue to ask questions according to the questions in the current environment until enough information is collected, i.e., realize the construction of the interview system.

2. The interview system construction method based on the large language model multi-agent mechanism according to claim 1, characterized in that: Also includes step 2.4: test the designed interview system; Inviting relevant personnel to use the interviewer model for interview experience, improving relevant deficiencies according to personnel feedback, and improving the interview system experience.

3. The interview system construction method based on the large language model multi-agent mechanism according to claim 2, characterized in that: It also includes step three, comparing and evaluating the interview records obtained by the interview system to obtain evaluation results. Step 3.1: Establish evaluation criteria for comparative evaluation. To evaluate the interview records obtained by the interview system, judge the consistency of the job matching degree reflected by the interview records, and judge the effectiveness of the system; The evaluation criteria are: (1) Professional matching degree: whether the professional skills of the interviewee match the job requirements; (2) Education experience matching degree: whether the education experience of the interviewee matches the job requirements; Step 3.2: Invite enterprise HR and use large model scoring; Obtain 100 interview process records, invite enterprise HR to score the interview process records; At the same time, use the large model to score the interview process records, and record two scores. Step 3.3: Compare and calculate the internal consistency and correlation of the scores of the large model and enterprise HR; Cronbach's Alpha coefficient and McDonald's omega coefficient are calculated for the two groups of scores, and both Cronbach's Alpha coefficient and McDonald's omega coefficient are greater than 0.8, indicating that the internal consistency meets the preset requirements; Pearson correlation coefficient and Spearman rank correlation coefficient are calculated for the two groups of scores, and both Pearson correlation coefficient and Spearman rank correlation coefficient are greater than 0.8, indicating high correlation.

4. The interview system construction method based on the large language model multi-agent mechanism according to claim 1, 2 or 3, characterized in that: Apply the interview system to the fields of enterprise recruitment, career development services, online education platforms, government talent selection, higher education institutions, and customer service industries to interactively generate interview questions and improve the service quality and efficiency of the above processes; In enterprise recruitment, the interview system is integrated into the enterprise recruitment process, automating the preliminary screening process, quickly evaluating the communication skills and professional skills of candidates through simulated interviews, reducing the workload of human resources departments, and improving the efficiency and quality of the entire interview process; In career development services, the interview system serves as a personalized interview training tool for job seekers, providing real-time feedback and improvement suggestions to help job seekers quickly and accurately correct pronunciation errors and improve pronunciation levels, thereby increasing their chances of obtaining a position; In online education platforms, the interview system serves as part of students' career development, helping students adapt to the pressure of real interviews in advance by simulating interview environments, providing real-time feedback, and enhancing students' interview skills; In the government talent selection process, the interview system serves as a fair and transparent auxiliary tool to help evaluate the comprehensive abilities of candidates and ensure the fairness of the selection process; In higher education institutions, university career service centers use the interview system to help graduating students prepare for interviews, improve their employment rates, and enhance their career preparation levels; In the customer service industry, the interview system is used to screen and train customer service representatives with excellent communication skills, thereby improving customer satisfaction and enterprise service quality.

Images