AI large model-based auxiliary interview method and apparatus, and electronic device

Abstract

The invention provides an auxiliary interview method and device based on an AI large model and electronic equipment. The method comprises the following steps: inputting a plurality of candidate resume sets and a post screening rule set into a fine-tuned large language model for resume screening to obtain a plurality of target resumes and a matching degree score of each target resume; in response to the screened target resume, starting an AI video interview for the interviewer in the target resume, obtaining a real-time video interview flow, and determining a multi-dimensional video interview score based on the real-time video interview flow; determining a professional skill score of each interviewer based on the question and answer content of each interviewer in the AI video interview and a pre-constructed knowledge graph of a target domain; and determining the post matching degree of each interviewer based on the matching degree score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine an interview passing person according to the post matching degree. According to the invention, the matching precision of the interviewer and the post is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an auxiliary interview method, device and electronic equipment based on an AI big model. Background Art

[0002] In today's digital age, companies are faced with a massive database of resumes. It has become a difficult and costly task to accurately identify candidates that meet job requirements. This problem is particularly prominent in campus recruitment scenarios. The resumes of candidates for campus recruitment are highly homogeneous, causing the human resources team or hiring department to spend a lot of time and energy manually screening resumes and organizing interviews. The entire process is extremely inefficient.

[0003] Although some existing technologies have used artificial intelligence technology to help improve efficiency, such as AI video interviews, the interview assessment is limited to a single dimension, resulting in low accuracy in matching interviewees with positions. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide an assisted interview method, device and electronic device based on an AI big model to improve the interview efficiency and the matching accuracy between the interviewee and the position.

[0005] In a first aspect, an assisted interview method based on an AI big model is provided, the method comprising: Input several candidate resume sets and job screening rule sets into the fine-tuned large language model to screen resumes and obtain multiple target resumes and the matching score of each target resume; In response to the screened target resumes, an AI video interview is initiated for the interviewee in the target resume, and a real-time video interview stream is obtained, and a multi-dimensional video interview score is determined based on the real-time video interview stream, where the multi-dimensional video interview score includes at least: an emotion score, a body score, and an image score; Determine the professional skill score of each interviewee based on the questions and answers of each interviewee in the AI ​​video interview and the pre-built knowledge graph of the target field; The job matching degree of each interviewee is determined based on the matching score of each target resume, the multi-dimensional video interview score and the professional skills score, so that the candidates who pass the interview can be determined based on the job matching degree.

[0006] Optionally, the process of screening target resumes in the fine-tuned large language model includes: Capture the key information of each candidate resume and form a label dataset of the candidate resume based on the key information. The label dataset includes at least years of work experience, skill keyword set, school ranking, and average grade point average of professional courses. Determine the matching score of each label in the label data set of the candidate resume based on the job screening rule set, where the matching score of each label at least includes a skill matching score, an experience matching score, and an education matching score; The matching score of each candidate resume is determined based on the matching score of each tag and the resume matching score calculation formula; the resume matching score formula is:

[0007] In the formula, For the A labeled dataset of candidate resumes; The weights corresponding to the skill, experience, and education dimensions respectively; They are skill matching score, experience matching score and education matching score; Sort the candidate resumes in descending order according to their matching scores; and determine the top preset number of resumes as target resumes.

[0008] Optionally, determining the skill matching score of the candidate resume based on the job screening rule set includes: Obtain the skill set required for the position in the position screening rule set and the skill keyword set in the candidate resume label dataset; The skill keyword set and the skill set required for the position are input into the skill matching score calculation formula to calculate the skill matching score. The skill matching score calculation formula is:

[0009] In the formula, Matching scores for skills; is the skill keyword weight; Indicates the skill set required for the position; A set of skill keywords representing candidate resumes; is an indicator function. If the skill keyword set includes the skills required for the position, the value is 1, otherwise it is 0.

[0010] Optionally, determining a sentiment score based on the real-time video interview stream includes: Based on the pre-trained key point detection model, the key point coordinate information of the facial muscles in each frame image in the real-time video interview stream is extracted and composed into a key point sequence ,in Indicates The set of facial keypoint coordinates of the frame; Determine the facial action activity of each frame image based on the key point sequence; The facial key point coordinates of each frame image and its corresponding facial action activity are combined into a facial feature vector ; The facial feature vector of each frame image Input into the pre-trained emotion classification model for classification to obtain the emotion classification label of each frame image; The emotion score is calculated based on the emotion classification label of each frame image and the preset emotion score calculation model.

[0011] Optionally, the emotion classification model is composed of a convolutional neural network (CNN) and a recurrent neural network (RNN), which transforms the facial feature vectors of each frame image into The emotion classification labels of each frame image obtained by inputting into the pre-trained emotion classification model include: Extract facial feature vectors using convolutional neural network CNN spatial characteristics; The spatial features are transmitted to the hidden layer of the recurrent neural network RNN ​​to capture the temporal continuity of the facial feature vector and obtain the spatiotemporal features of the facial feature vector. The dynamic update formula is:

[0012] Where: is the weight matrix between hidden layers, which is used to indicate the strength of information flow between hidden layers; is the weight matrix from input features to hidden layer; is the bias vector of the hidden layer, which is used to fine-tune the basic state of the hidden layer; After the spatiotemporal features are input into the fully connected layer and normalization layer of the RNN for normalization, the emotion classification label of each frame image is output.

[0013] Optionally, determining a physical score based on the real-time video interview stream includes: Based on the pre-trained human posture estimation model, the coordinate information of the human joint points in each frame image in the real-time video interview stream is extracted and composed into a posture sequence ,in, Indicates The pose set of the frame; Determine the multi-dimensional features of limbs based on the posture sequence, and the multi-dimensional features of limbs include at least: arm swing frequency , body forward lean angle ; The limb score is determined based on the multi-dimensional characteristics of the limb and the preset limb score calculation model.

[0014] Optionally, determining the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-constructed knowledge graph of the target field includes: Extract professional terminology entities based on the questions and answers of each interviewer in the AI ​​video interview and form a professional terminology entity set; Taking each professional term entity as the query condition, search for entities associated with each professional term entity in the pre-built knowledge graph of the target domain to form an associated entity set, and determine the relationship weight between each professional term entity and the associated entity; The score of each professional term entity is calculated based on the associated entity set, relationship weight and the preset professional term entity score calculation formula; the professional term entity score calculation formula is:

[0015] in, It is a professional term entity; is an associated entity; is the associative entity set; is the knowledge graph of the target domain; is the relationship weight between the professional term entity and the associated entity; is an indicator function, which is 1 if the professional term entity is found in the knowledge graph, otherwise it is 0; The professional skill score is calculated by weighted summation based on the scores of each professional term entity.

[0016] Optionally, professional terminology entities are extracted based on the questions and answers of each interviewer in the AI ​​video interview, and a professional terminology entity set is formed, including: Obtain the voice information of each interviewer during the AI ​​video interview; Convert voice information into a sequence of answer text records ; Performing text preprocessing on the answer text record sequence, the text preprocessing includes but is not limited to: removing stop words and punctuation marks; Use the word segmentation tool to segment the preprocessed answer text record sequence to obtain the word sequence ; Use the pre-trained word vector model to generate word vector sequences from word sequences ,in Expressive words The vector representation of ; Input the word vector sequence into the pre-trained BERT model to obtain the text semantic feature vector ; Identify the professional terminology entities in the semantic feature vector and obtain the professional terminology entity set as .

[0017] In a second aspect, an auxiliary interview device based on an AI large model is provided, the device comprising: A resume screening unit is used to input a plurality of candidate resume sets and a job screening rule set into the fine-tuned large language model to screen resumes and obtain a plurality of target resumes and a matching score for each target resume; A first determination unit is configured to, in response to the screened target resume, initiate an AI video interview for the interviewee in the target resume, obtain a real-time video interview stream, and determine a multi-dimensional video interview score based on the real-time video interview stream, wherein the multi-dimensional video interview score includes at least: an emotion score, a body score, and an image score; The second determination unit is used to determine the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-built knowledge graph of the target field; The third determination unit is used to determine the job matching degree of each interviewee based on the matching score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the interview passers-by according to the job matching degree.

[0018] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0019] According to a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0020] The present invention provides an auxiliary interview method, device and electronic device based on an AI large model. The method obtains multiple target resumes and the matching score of each target resume by inputting several candidate resume sets and job screening rule sets into a fine-tuned large language model for resume screening; in response to the screened target resume, an AI video interview is started for the interviewee in the target resume, and a real-time video interview stream is obtained, and a multi-dimensional video interview score is determined based on the real-time video interview stream; the professional skill score of each interviewee is determined based on the question and answer content of each interviewee in the AI ​​video interview and the knowledge graph of the pre-constructed target field; the job matching of each interviewee is determined based on the matching score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the interview passer according to the job matching. The present invention can help quickly screen resumes through a fine-tuned large language model, and can combine AI video interviews to evaluate the external state and internal ability of the interviewee from multiple aspects such as image, emotion, body, and professional skills. In addition, when evaluating professional skills, the knowledge graph of the industry is also combined for evaluation, which further improves the accuracy of the interview evaluation, thereby improving the matching accuracy between the interviewee and the position.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of an AI big model-based assisted interview method provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of an AI large model-based auxiliary interview device provided in an embodiment of the present invention is shown; Figure 3 A structural diagram of a + patent system name provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0025] Considering the low efficiency of resume screening and the low accuracy of matching the interviewer with the position, the embodiment of the present invention provides an auxiliary interview method and device based on an AI big model, which is described below through an embodiment.

[0026] The embodiment of the present invention provides an assisted interview method based on an AI big model, such as Figure 1 As shown, the method comprises the following steps: Step S101: Input several candidate resume sets and job screening rule sets into the fine-tuned large language model to screen resumes and obtain multiple target resumes and a matching score for each target resume.

[0027] In the embodiment of the present invention, the large language model uses an open source large language model, such as ChatGLM-6B or ChatGLM2-6B. With the large language model as the basic model, these models have been pre-trained on a large amount of text and can capture language features well. Using recruitment data in a specific field, the model is fine-tuned, that is, training is continued on the basis of pre-training, so that the large language model is more suitable for the tasks of the recruitment scenario.

[0028] In the embodiment of the present invention, the fine-tuning training of the large language model mainly includes two parts of training. One part is label classification of resumes; the other part is scoring and sorting the resumes after label classification to screen out resumes that meet the requirements.

[0029] In one possible implementation, the fine-tuning process for the large language model is as follows: Step 1: Prepare training sample data.

[0030] The training sample data includes the company's historical recruitment data and the company's job description document set ; Among them, historical recruitment data This includes data such as successful recruitment cases, job descriptions, and resumes of high-quality candidates from the past. For example, for software development positions, the resume information of a large number of outstanding programmers, interview evaluations at the time of joining the company, and daily performance feedback are used as training materials to allow the model to accurately grasp the key skills required for the position, the characteristics of project experience description, etc. It includes key contents such as the responsibilities of each position, essential skill requirements, and basic qualifications for the position.

[0031] Step 2: Historical recruitment data of the company Add labels, for example, skill labels, experience levels, etc.

[0032] The third step is to use the labeled historical recruitment data and the company's job description document set The input is fed into a large language model for fine-tuning training, which first maps them to a classification problem. Using the cross entropy loss function To measure the model output and the true label The difference between Updates.

[0033] The cross entropy loss function as follows: (1); Where: Points to the total number of training samples; The number of sample label categories, such as detailed classification of skill labels and precise grading of experience levels; Representative samples Corresponding category The actual calibration result, the value is 0 or 1; The model is for the sample Belong to category The estimated probability of .

[0034] The large language model then uses the job description document set to train the matching degree of each resume.

[0035] Step 4: Apply optimization algorithms such as stochastic gradient descent (SGD) or its variants such as Adam to adjust parameters to ensure that the model can effectively learn useful information from the data.

[0036] After the training is completed, the candidate resumes and job screening rule sets can be input into the large language model, and the screened target resumes and the matching scores of the target resumes can be directly output.

[0037] Step S102: In response to the screened target resume, an AI video interview is initiated for the interviewee in the target resume, and a real-time video interview stream is obtained, and a multi-dimensional video interview score is determined based on the real-time video interview stream.

[0038] The multi-dimensional video interview scores include at least: emotion score, body score, and image score.

[0039] In an embodiment of the present invention, an AI interviewer's AI image can be set to simulate a real interviewer for an interview. The target resume is input into the AI ​​interviewer's system. After reading the resume, the AI ​​interviewer will examine and evaluate the interviewee's image, body language, emotions, etc., and ask questions and follow-up questions.

[0040] Specifically, image recognition technology can be used to identify the image of the interviewee. For example, facial recognition can be used to determine whether the interviewee has tattoos or dyed hair, facial muscle recognition can be used to determine the interviewee's emotions, and body recognition can be used to determine whether the interviewee's sitting posture is correct.

[0041] The evaluation of body language and emotions will be described in the following embodiments and will not be elaborated here.

[0042] During the questioning process, the interviewee can also be evaluated from other dimensions, such as job competency, psychological quality, cognitive ability, career choice preferences, etc., and the interview results of each dimension can be quantified through scores.

[0043] Through multi-dimensional evaluation, the matching accuracy between automated interview results and positions can be greatly improved.

[0044] Step S103: Determine the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-constructed knowledge graph of the target field.

[0045] During the interview process, the AI ​​interviewer obtains the interviewee's responses to questions and follow-up questions, and determines the interviewee's professional skills based on the interviewee's responses. In the embodiment of the present invention, the professional terms of each assessment dimension involved in the interviewee's answers are deeply analyzed in combination with the industry knowledge graph, thereby improving the accuracy of identifying the interviewee's industry skills and industry experience.

[0046] The specific algorithm for determining professional skill scores using the knowledge graph will be described in the following embodiments and will not be repeated here.

[0047] Step S104: Determine the job matching of each interviewee based on the matching score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the interviewee who passes the interview based on the job matching score.

[0048] In this step, the job matching degree can be calculated by the following formula: , the calculation formula is as follows: (2); In the formula, Score the resume match; Score emotions; score points for limbs; Score points for professional skills; are the weights of the scores of each dimension.

[0049] After calculating the job matching degree of each interviewee, you can select the candidates through the preset interview. To quickly screen those who have passed the interview, if , then you will be recommended to enter the next round of interviews, otherwise you will fail the interview.

[0050] In addition, you can also set the focus points ;like , then mark the interviewee as a key focus object.

[0051] In a feasible implementation, an evaluation report can be generated for those who have passed the interview. The report can incorporate elements such as text analysis and visual charts (such as score radar charts to intuitively compare performance differences in various dimensions) to provide accurate reference for corporate HR and business departments, thereby facilitating efficient talent decision-making.

[0052] The embodiment of the present invention can help quickly screen resumes through a fine-tuned large language model, and can combine with AI video interviews to evaluate the external state and internal ability of the interviewee from multiple aspects such as image, emotion, body language, and professional skills. In addition, when evaluating professional skills, it is also combined with the industry's knowledge graph to further improve the accuracy of the interview evaluation, thereby improving the matching accuracy between the interviewee and the position.

[0053] Based on the above embodiment, the process of screening target resumes includes: Step S101A: Capture key information of each candidate resume, and compose a label data set of the candidate resume based on the key information.

[0054] In this step, the label data set includes at least years of work experience, skill keyword set, school ranking, and average grade point average of professional courses; Step S101B: determining the matching score of each tag in the tag data set of the candidate resume based on the job screening rule set, where the matching score of each tag at least includes a skill matching score, an experience matching score, and an education matching score; Step S101C: Determine the matching score of each candidate resume based on the matching score of each tag and the resume matching score calculation formula; the resume matching score formula is: (3); In the formula, For the A labeled dataset of candidate resumes; The weights corresponding to the skill, experience, and education dimensions respectively; They are skill matching score, experience matching score and academic qualification matching score.

[0055] Step S101D: Sort the candidate resumes in descending order according to their matching scores; and determine the resumes that are ranked in the front by a preset number as target resumes.

[0056] In this step, for example, if the preset number is 3, the top 3 resumes will be determined as target resumes.

[0057] The embodiment of the present invention screens resumes from multiple dimensions such as education background, experience, and skills, thereby improving the matching accuracy between the screened resumes and positions.

[0058] In a feasible implementation, determining the skill matching score of a candidate resume based on a job screening rule set includes: Obtain the skill set required for the position in the position screening rule set and the skill keyword set in the candidate resume label dataset; The skill keyword set and the skill set required for the position are input into the skill matching score calculation formula to calculate the skill matching score. The skill matching score calculation formula is: (4); In the formula, Matching scores for skills; is the skill keyword weight; Indicates the skill set required for the position; A set of skill keywords representing candidate resumes; is an indicator function. If the skill keyword set includes the skills required for the position, the value is 1, otherwise it is 0.

[0059] The experience matching score and education matching score can be calculated in the same way. For example, for the experience matching score, according to the experience required in the job screening rule set, for example, 3 years, and the working years in the label data set is 1 year, the experience matching score calculation formula can be used to calculate the experience matching score. The experience matching score calculation formula can refer to formula 4, and only the corresponding data needs to be replaced with the working years and the experience data required for the position. The education matching score can be calculated in the same way, which will not be repeated here.

[0060] Based on the above embodiment, determining the emotion score based on the real-time video interview stream includes: Step A: Extract the key point coordinate information of facial muscles in each frame of the real-time video interview stream based on the pre-trained key point detection model and form a key point sequence ,in Indicates A collection of facial keypoint coordinates for a frame.

[0061] In one example, these key points may include specific locations of eyebrows, eyes, nose, mouth, etc.

[0062] In this step, the key point detection model adopts a deep learning model, the training samples can be facial images composed of various emotions, the loss function of the model can adopt a cross-entropy loss function, and the training process is the same as the conventional training method of the deep learning model, which will not be repeated here.

[0063] Step B: Determine the facial action activity of each frame image based on the key point sequence.

[0064] In this step, for each frame, the facial action activity of the frame is calculated based on the set of key point coordinates. This can be done by measuring the relative displacement, angle change or other geometric property changes between key points. The activity can reflect the degree of change in facial expression, such as the stretching of the corners of the mouth when smiling or the movement of the eyebrows when frowning.

[0065] Step C: Combine the facial key point coordinates of each frame image with its corresponding facial action activity into a facial feature vector .

[0066] This vector can capture information about facial structure and dynamic changes, providing rich input data for subsequent emotion classification.

[0067] Step D: Transform the facial feature vectors of each frame image Input into the pre-trained emotion classification model for classification to obtain the emotion classification label of each frame image.

[0068] In this step, the emotion classification model is composed of a convolutional neural network (CNN) and a recurrent neural network (RNN). The training samples use facial images composed of various emotions. The loss function of the model can use a cross-entropy loss function. The training process is the same as the conventional training method of the neural network and will not be repeated here.

[0069] In a feasible implementation, the facial feature vectors of each frame image are The emotion classification labels of each frame image obtained by inputting into the pre-trained emotion classification model include: Step D1: Extract facial feature vector using convolutional neural network (CNN) spatial characteristics; Step D2: The spatial features are transmitted to the hidden layer of the recurrent neural network RNN ​​to capture the temporal continuity of the facial feature vector and obtain the spatiotemporal features of the facial feature vector. The dynamic update formula is: (5); Where: is the weight matrix between hidden layers, which is used to indicate the strength of information flow between hidden layers; is the weight matrix from input features to hidden layer; is the bias vector of the hidden layer, which is used to fine-tune the basic state of the hidden layer; Step D3: After the spatiotemporal features are input into the fully connected layer and normalization layer of the RNN for normalization, the emotion classification label of each frame image is output.

[0070] In this step, the emotion classification labels are, for example, nervous, confident, excited, etc.

[0071] Step E: Calculate the emotion score based on the emotion classification label of each frame image and the preset emotion score calculation model.

[0072] In this step, the preset score calculation model is: (6); In the formula, Preset basic scores for each emotion tag, such as confidence = 5 points, nervousness = -3 points, etc. The confidence level for each sentiment label.

[0073] Based on the above embodiment, determining the limb score based on the real-time video interview stream includes: Step a: Based on the pre-trained human posture estimation model, the coordinate information of the human joint points in each frame image in the real-time video interview stream is extracted and composed into a posture sequence ,in, Indicates A collection of poses for a frame.

[0074] In this step, the human posture estimation model can adopt models such as enPose, HRNet, AlphaPose, etc. The model can detect key points of the human body such as shoulders, elbows, wrists, hips, knees and ankles.

[0075] Step b: Determine the multi-dimensional features of the limbs based on the posture sequence, and the multi-dimensional features of the limbs include at least: arm swing frequency , body forward lean angle .

[0076] In this step, since real-time video may introduce noise, it is possible to consider smoothing the pose sequence, for example, by reducing jitter through a moving average filter.

[0077] In a feasible implementation manner, calculating the arm swing frequency includes the following steps: 1. Select reference points: Select the shoulder and wrist as reference points to determine the position of the arm. Assume there are key points for the left and right shoulders And the key points of the left and right wrists .

[0078] 2. Define the arm position: Use the direction vector of the line connecting the shoulder and wrist to represent the arm position. For example, for the left arm, you can use the vector express.

[0079] 3. Calculate the swing period: Swing events are detected by the change in arm position between consecutive frames. A swing is considered complete when the arm moves from one side to the other. The start and end of a swing can be determined by setting a threshold to determine whether the arm has crossed the center line (i.e., the line perpendicular to the shoulder).

[0080] 4. Frequency calculation: Count the number of swings within a fixed time window, and then divide it by the time length to get the frequency swing frequency. If you need to calculate the frequencies of the left and right hands separately, repeat the above steps for each hand.

[0081] In one possible implementation, the body forward lean angle is calculated include: 1. Select key points: Select key points on the spine, such as the neck and hips , or more specifically, the key points of the upper and lower back to construct a vector representing the torso.

[0082] 2. Create a reference frame: Create a reference coordinate system where the Y axis is the vertical direction. This is usually based on the first frame of the video or a known vertical reference.

[0083] It should be noted that if the camera is not fixed or the interviewer is moving, the reference coordinate system also needs to be updated accordingly to maintain accuracy.

[0084] 3. Calculate the angle: use vector To represent the main axis of the body, and calculate the angle between this vector and the Y axis in the reference coordinate system This angle can be calculated using the inverse tangent function atan2:

[0085] in, , They are vectors Components in the horizontal and vertical directions.

[0086] Step c: Determine the limb score based on the multi-dimensional characteristics of the limb and the preset limb score calculation model.

[0087] In this step, since different limb features may have different dimensions and ranges (for example, frequency is several times per second, while angle may be degrees), these features need to be standardized or normalized before calculation. This step ensures that different features are comparable during the calculation process. In one feasible implementation, each feature value is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1 through Z-Score normalization; using normalization, the feature value is scaled to a specific interval .

[0088] The preset limb score calculation model is as follows: (8); in, It is the limb score; It is The weight of each feature; It is Body feature values.

[0089] Based on the above embodiment, determining the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-constructed knowledge graph of the target field includes: Step S103A: Extracting professional terminology entities based on the questions and answers of each interviewer in the AI ​​video interview and forming a professional terminology entity set .

[0090] Step S103B: Using each professional term entity as a query condition, search for entities associated with each professional term entity in the pre-constructed knowledge graph of the target domain to form an associated entity set, and determine the relationship weight between each professional term entity and the associated entity.

[0091] Step S103C: Calculate the score of each professional term entity based on the associated entity set, the relationship weight and the preset professional term entity score calculation formula; the professional term entity score calculation formula is: (9); in, It is a professional term entity; is an associated entity; is the associative entity set; is the knowledge graph of the target domain; is the relationship weight between the professional term entity and the associated entity; is an indicator function, which is 1 if the professional term entity is found in the knowledge graph, otherwise it is 0; Step S103D: Obtain a professional skill score by weighted summation based on the scores of each professional term entity.

[0092] The embodiment of the present invention can accurately identify the professional terms in the interviewee's response content and accurately evaluate his / her mastery of professional skills by combining the knowledge graph, thereby improving the accuracy of the entire interview result and the job matching.

[0093] Based on the above embodiment, professional terminology entities are extracted based on the questions and answers of each interviewer in the AI ​​video interview, and a professional terminology entity set is formed, including: Obtain the voice information of each interviewer during the AI ​​video interview; Convert voice information into a sequence of answer text records ; Performing text preprocessing on the answer text record sequence, the text preprocessing includes but is not limited to: removing stop words and punctuation marks; Use the word segmentation tool to segment the preprocessed answer text record sequence to obtain the word sequence ; Use the pre-trained word vector model to generate word vector sequences from word sequences ,in Expressive words The vector representation of ; Input the word vector sequence into the pre-trained BERT model to obtain the text semantic feature vector ; Identify the professional terminology entities in the semantic feature vector and obtain the professional terminology entity set as .

[0094] Based on the same inventive concept, an auxiliary interview device based on an AI big model is provided, such as Figure 2 As shown, the device comprises: A resume screening unit 201 is used to input a plurality of candidate resume sets and a job screening rule set into the fine-tuned large language model to screen resumes and obtain a plurality of target resumes and a matching score for each target resume; A first determining unit 202 is configured to initiate an AI video interview with an interviewee in the target resume in response to the screened target resume, obtain a real-time video interview stream, and determine a multi-dimensional video interview score based on the real-time video interview stream, where the multi-dimensional video interview score includes at least an emotion score, a body score, and an image score; The second determination unit 203 is used to determine the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-constructed knowledge graph of the target field; The third determination unit 204 is used to determine the job matching degree of each interviewer based on the matching degree score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the interview passers according to the job matching degree.

[0095] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, such as Figure 3 As shown, it includes a processor 301 , a communication interface 302 , a memory 303 and a communication bus 304 , wherein the processor 301 , the communication interface 302 , and the memory 303 communicate with each other via the communication bus 304 .

[0096] Memory 303, used for storing computer programs; The processor 301 is used to implement the steps of the assisted interview method based on the AI ​​big model when executing the program stored in the memory 303.

[0097] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0098] The communication interface is used for communication between the above electronic device and other devices.

[0099] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0100] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0101] The computer program product for the AI ​​big model-based assisted interview method provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0102] The device for assisted interview based on AI big model provided in the embodiment of the present invention can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effect of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0103] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0104] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, each functional unit in the embodiment provided by the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0106] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0107] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0108] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An assisted interview method based on an AI big model, characterized in that: The method comprises: Input several candidate resume sets and job screening rule sets into the fine-tuned large language model to screen resumes and obtain multiple target resumes and the matching score of each target resume; In response to the screened target resume, an AI video interview is initiated for the interviewee in the target resume, and a real-time video interview stream is obtained, and a multi-dimensional video interview score is determined based on the real-time video interview stream, wherein the multi-dimensional video interview score includes at least: an emotion score, a body score, and an image score; Determine the professional skill score of each interviewee based on the questions and answers of each interviewee in the AI ​​video interview and the pre-built knowledge graph of the target field; The job matching degree of each interviewee is determined based on the matching score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the candidates who pass the interview based on the job matching degree.

2. The method according to claim 1, characterized in that The process of screening target resumes in the fine-tuned large language model includes: Capture key information of each candidate resume, and form a label data set of the candidate resume based on the key information, wherein the label data set at least includes years of work experience, skill keyword set, school ranking, and average grade point average of professional courses; Determine the matching score of each tag in the tag data set of the candidate resume based on the job screening rule set, wherein the matching score of each tag includes at least a skill matching score, an experience matching score and an education matching score; The matching score of each candidate resume is determined based on the matching score of each tag and the resume matching score calculation formula; the resume matching score calculation formula is: In the formula, For the A labeled dataset of candidate resumes; The weights corresponding to the skill, experience, and education dimensions respectively; They are skill matching score, experience matching score and education matching score; Sort the candidate resumes in descending order according to their matching scores; and determine the top preset number of resumes as target resumes.

3. The method according to claim 2, characterized in that The skill matching scores of candidate resumes are determined based on the job screening rule set, including: Obtain the skill set required for the position in the position screening rule set and the skill keyword set in the candidate resume label dataset; The skill keyword set and the skill set required for the position are input into the skill matching score calculation formula to calculate the skill matching score. The skill matching score calculation formula is: In the formula, Matching scores for skills; is the skill keyword weight; Indicates the skill set required for the position; A set of skill keywords representing candidate resumes; is an indicator function. If the skill keyword set includes the skills required for the position, the value is 1, otherwise it is 0.

4. The method according to claim 1, characterized in that Determining a sentiment score based on the live video interview stream includes: Based on the pre-trained key point detection model, the key point coordinate information of the facial muscles in each frame image in the real-time video interview stream is extracted and formed into a key point sequence ,in Indicates The set of facial keypoint coordinates of the frame; Determining the facial action activity of each frame image based on the key point sequence; The facial key point coordinates of each frame image and its corresponding facial action activity are combined into a facial feature vector ; The facial feature vector of each frame image Input into the pre-trained emotion classification model for classification to obtain the emotion classification label of each frame image; The emotion score is calculated based on the emotion classification label of each frame image and a preset emotion score calculation model.

5. The method according to claim 4, characterized in that The emotion classification model is composed of a convolutional neural network (CNN) and a recurrent neural network (RNN). The emotion classification labels of each frame image obtained by inputting into the pre-trained emotion classification model include: Extract facial feature vectors using convolutional neural network CNN spatial characteristics; The spatial features are transmitted to the hidden layer of the recurrent neural network RNN ​​to capture the temporal continuity of the facial feature vector, and the spatiotemporal features of the facial feature vector are obtained, wherein the internal hidden state of the recurrent neural network RNN The dynamic update formula is: Where: is the weight matrix between hidden layers, which is used to indicate the strength of information flow between hidden layers; is the weight matrix from input features to hidden layer; is the bias vector of the hidden layer, which is used to fine-tune the basic state of the hidden layer; After the spatiotemporal features are input into the fully connected layer and normalization layer of the RNN for normalization operation, the emotion classification label of each frame image is output.

6. The method according to claim 1, characterized in that Determining a limb score based on the real-time video interview stream includes: Based on the pre-trained human posture estimation model, the coordinate information of the human joint points in each frame image in the real-time video interview stream is extracted and composed into a posture sequence ,in, Indicates The pose set of the frame; Determine limb multi-features based on the posture sequence, the limb multi-features at least including: arm swing frequency , body forward lean angle ; The limb score is determined based on the limb multi-dimensional characteristics and a preset limb score calculation model.

7. The method according to claim 1, characterized in that Determining the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-constructed knowledge graph of the target field includes: Extract professional terminology entities based on the questions and answers of each interviewer in the AI ​​video interview and form a professional terminology entity set; Taking each professional term entity as the query condition, search for entities associated with each professional term entity in the pre-built knowledge graph of the target domain to form an associated entity set, and determine the relationship weight between each professional term entity and the associated entity; The score of each professional term entity is calculated based on the associated entity set, the relationship weight and the preset professional term entity score calculation formula; the professional term entity score calculation formula is: in, It is a professional term entity; is an associated entity; is the associative entity set; is the knowledge graph of the target domain; is the relationship weight between the professional term entity and the associated entity; is an indicator function, which is 1 if the professional term entity is found in the knowledge graph, otherwise it is 0; The professional skill score is calculated by weighted summation based on the scores of the professional terminology entities.

8. The method according to claim 7, characterized in that The extraction of professional terminology entities based on the question and answer content of each interviewer in the AI ​​video interview and the formation of a professional terminology entity set include: Obtain the voice information of each interviewer during the AI ​​video interview; Convert the voice information into a text record sequence of answers ; Performing text preprocessing on the answer text record sequence, wherein the text preprocessing at least includes: removing stop words and punctuation marks; Use the word segmentation tool to segment the preprocessed answer text record sequence to obtain the word sequence ; Use the pre-trained word vector model to generate a word vector sequence from the word sequence ,in Expressive words The vector representation of ; Input the word vector sequence into the pre-trained BERT model to obtain the text semantic feature vector ; Identify the professional terminology entities in the semantic feature vector and obtain the professional terminology entity set as .

9. An assisted interview device based on an AI big model, characterized in that: The device comprises: A resume screening unit is used to input a number of candidate resume sets and a job screening rule set into the fine-tuned large language model to screen resumes and obtain multiple target resumes and a matching score for each target resume; A first determination unit is configured to, in response to the screened target resume, initiate an AI video interview for the interviewee in the target resume, obtain a real-time video interview stream, and determine a multi-dimensional video interview score based on the real-time video interview stream, wherein the multi-dimensional video interview score includes at least: an emotion score, a body score, and an image score; The second determination unit is used to determine the professional skill score of each interviewee based on the question and answer content of each interviewee in the AI ​​video interview and the pre-built knowledge graph of the target field; The third determination unit is used to determine the job matching degree of each interviewer based on the matching score of each target resume, the multi-dimensional video interview score and the professional skill score, so as to determine the interview passers according to the job matching degree.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 8 when executing a program stored in a memory.

Images