Automatic interview method and device based on artificial intelligence, computer equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and discloses an automatic interview method and device based on artificial intelligence, computer equipment and a storage medium, and the method comprises the steps: obtaining keyword information for a delivery post in a resume file of a target object, and employing a large language model based on the keyword information and post requirements of the delivery post, generating a question for the target object; generating a corresponding questioning video based on the questioning question, obtaining the answer content of the target object for the questioning video, and determining an evaluation result corresponding to the answer content based on a preset evaluation standard by adopting a large language model; and generating a corresponding evaluation video based on the evaluation result to enable the target object to obtain the evaluation result, thereby realizing automatic generation of the question question and answer content evaluation result in the interview process, and interacting with the target object in a mode of generating the question video and the evaluation video, so that the interview process does not need an interviewer, and the interview efficiency is improved. Therefore, the automation degree of the interview process is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an automatic interview method, device, computer equipment and storage medium based on artificial intelligence. Background Art

[0002] With the advancement of technology and the development of the times, the number of employed people is increasing day by day, and companies will receive a large number of interview applications during the recruitment process; the current existing technology can assist companies in interviewing job seekers through artificial intelligence (AI); however, the existing AI-assisted interview methods have a low degree of automation, resulting in cumbersome and complex problems in the interview process, which requires a better solution. Summary of the invention

[0003] In view of this, the present invention provides an automatic interview method, apparatus, computer equipment and storage medium based on artificial intelligence to solve the problem of how to improve the degree of automation in the interview process.

[0004] On the one hand, the present invention provides an automatic interview method based on artificial intelligence, the method comprising obtaining keyword information for an applied position in a resume file of a target object, and based on the keyword information and the job requirements of the applied position, using a large language model to generate questions for the target object; the keyword information is used to characterize the degree of match between the target object and the applied position; a corresponding question video is generated based on the question, the target object's answer content for the question video is obtained, and the large language model is used to determine an evaluation result corresponding to the answer content based on a preset evaluation standard; based on the evaluation result, a corresponding evaluation video is generated so that the target object obtains the evaluation result.

[0005] On the other hand, the present invention also provides an automatic interview device based on artificial intelligence, which includes: a questioning module, which is used to obtain keyword information for the applied position in the resume file of the target object, and based on the keyword information and the job requirements of the applied position, use a large language model to generate questions for the target object; the keyword information is used to characterize the degree of match between the target object and the applied position; an evaluation module, which is used to generate a corresponding question video based on the question, obtain the target object's answer content to the question video, use the large language model, and determine the evaluation result corresponding to the answer content based on preset evaluation criteria; an interactive module, which is used to generate a corresponding evaluation video based on the evaluation result, so that the target object obtains the evaluation result.

[0006] On the other hand, the present invention also provides a computer device, which includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned automatic interview method based on artificial intelligence by executing the computer instructions.

[0007] On the other hand, the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned automatic interview method based on artificial intelligence.

[0008] In this process, the questions and answer content evaluation results are automatically generated during the interview, and the target object is interacted with by generating question videos and evaluation videos, so that the interview process does not require an interviewer, thereby improving the degree of automation of the interview process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 It is a flowchart of an automatic interview method based on artificial intelligence provided by an embodiment of the present invention;

[0011] Figure 2 This is a specific flow chart of an automatic interview method based on artificial intelligence provided by an embodiment of the present invention;

[0012] Figure 3 This is a schematic diagram of questions asked by an automatic interview method based on artificial intelligence provided by an embodiment of the present invention;

[0013] Figure 4 It is a schematic diagram of evaluation results of an automatic interview method based on artificial intelligence provided by an embodiment;

[0014] Figure 5 It is a structural schematic diagram of an automatic interview device based on artificial intelligence provided by an embodiment of the present invention;

[0015] Figure 6 It is a structural schematic diagram of another automatic interview device based on artificial intelligence provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In the current digital era, the traditional face-to-face interview method in the recruitment process of enterprises faces problems such as high time cost, high labor cost, and obvious geographical restrictions. With the rapid development of artificial intelligence technology, various companies are exploring an interview method assisted by AI; in existing technologies, AI can be used to assist in scoring the interviewee's answers, and AI can also be used to automatically generate the next question based on the interviewee's answer.

[0017] However, the existing solutions have the following problems:

[0018] First, the current AI-assisted interview method improves the automation level of a specific step in the interview process, but the automation level of the entire interview process is not high;

[0019] Second, AI-generated questions are more similar, and the scope of the interviewee is narrower;

[0020] Third, the AI-assisted interview process is a generative process, which has poor interactivity with the interviewees.

[0021] To solve at least one of the above problems, various embodiments of the present invention provide an automatic interview method based on artificial intelligence, including: obtaining keyword information for the applied position in the resume file of the target object, and using a large language model to generate questions for the target object based on the keyword information and the job requirements of the applied position; the keyword information is used to characterize the degree of match between the target object and the applied position; a corresponding question video is generated based on the question, the target object's answer content for the question video is obtained, and the large language model is used to determine the evaluation result corresponding to the answer content based on preset evaluation criteria; based on the evaluation result, a corresponding evaluation video is generated so that the target object obtains the evaluation result.

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] According to an embodiment of the present invention, an automatic interview method based on artificial intelligence is provided. Figure 1 is a flow chart of an automatic interview method based on artificial intelligence provided by an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0024] Step S101, obtaining keyword information for the applied position in the resume file of the target object, and using a large language model to generate questions for the target object based on the keyword information and the job requirements of the applied position; the keyword information is used to characterize the matching degree between the target object and the applied position;

[0025] Step S102, generating a corresponding question video based on the question, obtaining the target object's answer content to the question video, and using a large language model to determine an evaluation result corresponding to the answer content based on a preset evaluation standard;

[0026] Step S103: Based on the evaluation result, a corresponding evaluation video is generated so that the target object can obtain the evaluation result.

[0027] In a possible implementation, based on keyword information and job requirements of the job being applied for, a large language model is used to generate questions for the target object, which may include:

[0028] Read the keyword information in the resume file, use the large language model to generate questions for the job application, and ask questions to the target object.

[0029] Here, preset constraints can be imposed on the large language model so that the questions determined by the large model give priority to the technical points mentioned in the resume file. This can reduce the repetition of questions and make the technical assessment of the interviewee more targeted.

[0030] Specifically, the keyword information is ranked by importance, the priority used as the basis for generating questions is determined, and questions about the technical points mentioned in the resume file are generated preferentially according to the priority ranking in the keyword information; before generating questions, a deduplication mechanism is added to check the existing question history to ensure that the same or overly similar questions are not generated.

[0031] Exemplarily, corresponding questions can be generated based on the target object's job type; for example, if the target object's job type is a technician, the questions may include but are not limited to: "How to implement multi-threading in Java and avoid thread safety issues"; if the target object's job type is a salesperson, the questions may include but are not limited to: "How to maintain and enhance customer affinity during communication with customers."

[0032] Among them, the target object may refer to the interviewee, and the keyword information in the resume file may be used to characterize the professional skills, work experience, and educational background of the target object; the Large Language Model (LLM) may be a natural language processing (NLP) model based on deep learning, which is used to understand, generate, and process human language. Using a large language model to generate questions can ensure the objectivity of the interview process, improve the efficiency of the interview process, and reduce labor costs.

[0033] In a possible implementation, generating a corresponding question video based on the question can be implemented based on the following steps:

[0034] Text-to-Speech (TTS) technology is used to convert questions into speech, and digital human generation technology is used to generate digital human questioning videos.

[0035] Here, TTS technology can refer to a technology that converts text into natural speech output by reading the input text and converting it into sounds that simulate human speech; digital human generation technology can refer to a technology that uses computer graphics, NLP and AI to generate virtual human images and behaviors. By generating a virtual human image of an interviewer through digital human generation technology, asking questions to the interviewee and commenting on the interviewee's answers, the interactivity of the interview process can be improved, and more intuitive feedback results can be provided to the interviewee.

[0036] In one possible implementation, the preset evaluation criteria may refer to preset answer content evaluation criteria provided for the large language model. The evaluation dimensions of the preset evaluation criteria may include but are not limited to: answer accuracy, answer depth and breadth, expression logic, and answer innovation; the large language model scores the answer content based on the multi-dimensional evaluation criteria to generate an evaluation result.

[0037] Here, the representation form of the evaluation result can be specifically set according to needs in the prompt words before the large language model performs the evaluation; for example, the evaluation result can be expressed in percentage, or it can be expressed based on preset grades such as A, B, C, or excellent, good, or unqualified.

[0038] In a possible implementation, based on the evaluation result, a corresponding evaluation video is generated so that the target object obtains the evaluation result, including:

[0039] Obtain a text report corresponding to the evaluation results, and convert the text report into colloquial expressions to serve as the evaluation narration in the evaluation video;

[0040] The emotion perception model and the TTS model are used to generate emotional voice content based on the emotion types contained in the text report, and an evaluation video is generated based on the text report, voice content and digital human image.

[0041] Furthermore, a large language model may be used to generate visualization graphics corresponding to the evaluation results, and the evaluation video may be generated in combination with the visualization graphics; the visualization video may include but is not limited to: bar charts, radar charts, line charts, etc.

[0042] Through the above method, a large language model is used to generate questions that are highly relevant to the applied position, which improves the pertinence of interview questions and the degree of interview automation. By combining the large language model with TTS and video generation technology, question videos and evaluation videos are automatically generated, which saves a lot of human resources in the interview process and improves the interview processing capabilities.

[0043] In one embodiment, keyword information for the job position in the resume file of the target object is obtained, and based on the keyword information and the job requirements of the job position, a large language model is used to generate questions for the target object, including:

[0044] Obtain the resume file of the target object, use the large language model to perform keyword extraction on the resume file, and obtain keyword information in the resume file;

[0045] Determine whether the job position in the keyword information is consistent with the applied position. When the job position is consistent with the applied position, randomly determine the prompt word information in the preset prompt word library corresponding to the applied position based on the keyword information, combine the prompt word information with the preset template to obtain a prompt sentence, and based on the prompt sentence, use a large language model to generate questions with a word count less than the word count threshold; the keyword information includes but is not limited to: job position, project experience, and ability level.

[0046] Specifically, the delivery position of the target object is determined, and a preset prompt word library corresponding to the delivery position is searched to determine the prompt word group that meets the preset position type; the prompt word information is combined with the preset template to obtain a prompt sentence, so that the large language model generates a question that meets the preset template based on the prompt word information.

[0047] For example, the prompt phrases in the preset prompt word library may include but are not limited to front-end, back-end, server, and test; when the post is for front-end development, the front-end is used as a keyword to search through the preset prompt word library to determine the prompt phrases associated with the front-end.

[0048] In a possible implementation, at least one set of prompt words is prepared for prompt phrases corresponding to each job type, and after the job type of the interviewee is determined, at least one set of prompt words is randomly selected from the prompt phrases associated with the job type.

[0049] Exemplarily, when the job type is front-end, the prompt phrases corresponding to the front-end may include but are not limited to: "The current job position is front-end, please ask 5 questions about the basic issues of front-end development", "The current job position is front-end, please ask 5 questions about the programming language of front-end development".

[0050] Furthermore, enabling the large language model to generate questions based on the format of a preset template can facilitate subsequent parsing of the answers to the questions.

[0051] Furthermore, by using a large language model to generate questions with a word count less than a word count threshold, the questions generated by the large language model can be made as brief as possible, thereby facilitating the generation of subsequent question videos and reducing the waiting time of the target object.

[0052] In a possible implementation, the method further includes:

[0053] A large language model is used to generate multiple questions, which are then split and a corresponding question video is generated for each question to prevent the large language model from losing constraint content in multiple rounds of interaction.

[0054] Through the above method, by randomly selecting different prompt words for the same position type, the phenomenon of asking the same questions due to repeated prompt words can be reduced, and the interviewee's understanding of the position can be better examined.

[0055] In one embodiment, asking a question to generate a corresponding question video is achieved based on the following steps:

[0056] Using digital human generation technology, a corresponding digital interviewer image is generated based on the target object's resume file;

[0057] The text information of at least one question generated by the large language model is obtained, and the voice information corresponding to each question is determined by using text-to-speech technology. Based on the image of the digital interviewer, the text information and the voice information, a question video corresponding to each question is generated.

[0058] In a possible implementation, digital human generation technology is used to generate a corresponding digital interviewer image based on the resume file of the target object, including:

[0059] Pre-screen resumes before interviews and automatically identify digital images of interviewers that match the job you are applying for.

[0060] Automatically identify the ethnic information in the target subject's resume file and generate the corresponding digital image of the interviewer.

[0061] Specifically, resume files are pre-screened before the interview to determine the job positions of each target subject, and target subjects that meet the job types are screened out. A digital image of the interviewer is generated based on the target subject's job position, such as HR image, sales manager image, and technical manager image. The ethnic information in the target subject's resume file is automatically identified. If the target subject is a foreigner, a digital interviewer with a foreigner image can be generated, and the interview with the target subject is conducted in full English.

[0062] In a possible implementation, a question video corresponding to each question is generated based on the digital interviewer image, text information, and question voice information, including:

[0063] Using a voice model with emotional expression, the generated text of each question is converted into voice information of the question. The voice style of the voice information of the question can be adjusted accordingly based on the formality of the interview so that the voice information of the question is consistent with the image of the digital human interviewer; the voice style may include but is not limited to: speaking speed, intonation, and tone;

[0064] Use lip synchronization technology to align the voice information of the question with the lip shape of the digital interviewer;

[0065] The image and facial expressions of the digital human interviewer are integrated with the question voice generated by TTS to generate a question video corresponding to each question.

[0066] Among them, speech models with emotional expression may include but are not limited to: Tacotron 2 model and WaveNet model; Tacotron 2 model can generate high-quality speech with natural intonation, pauses, and accents, and can imitate the subtle features of human speech; WaveNet model can model the details, timbre, pitch and accent of speech; lip synchronization technology is a technology that accurately matches speech with the lip movements of a digital person or virtual character.

[0067] Through the above method, TTS technology is used to convert questions into question voices, which can improve the clarity of the voice and reduce the understanding burden of the interviewer; digital human interviewers are generated through digital human generation technology, which can improve the immersion and interactivity of the interview process.

[0068] In one embodiment, the answer content of the target object to the question video is obtained, and the evaluation result corresponding to the answer content is determined based on the preset evaluation criteria using a large language model; the preset evaluation criteria are determined based on keyword information, and are implemented based on the following steps:

[0069] Using speech recognition technology to obtain the target object's answering voice, and using speech-to-text technology to obtain the answering text information corresponding to the answering voice;

[0070] Based on the ability level in the keyword information of the target object, the corresponding evaluation level is determined; the evaluation levels include: first level, second level and third level;

[0071] Evaluation prompt words are determined based on preset evaluation standards and evaluation levels. The evaluation prompt words are used to assemble prompt words on the answer text information. A large language model is used to generate an evaluation result corresponding to the answer content. The number of words in the evaluation result is less than the word count threshold.

[0072] In one possible implementation, speech recognition technology is used to obtain the target object's answer speech, including: for different job types, a corresponding professional terminology database is established, including the terms, keywords and phrases commonly used in the job field; for each job type, the professional terminology database of the corresponding field is used to fine-tune the speech recognition model, so that the model can more accurately understand and transcribe the target object's answer content in the specific job field.

[0073] For example, for software development positions, the professional terminology database may include programming languages, development frameworks, software engineering concepts, etc.; for sales positions, the professional terminology database may include sales skills, marketing, contract signing, etc.

[0074] In a possible implementation, determining a corresponding evaluation level based on the ability level in the keyword information of the target object includes:

[0075] The proficiency level of the work skills in the resume file is obtained. When the proficiency level is proficient, the corresponding assessment level is determined to be the first level. When the proficiency level is proficient, the corresponding assessment level is determined to be the second level. When the proficiency level is understood, the corresponding assessment level is determined to be the third level.

[0076] Here, proficiency can mean that the interviewee can tune the questions asked, proficiency can mean that the interviewee understands the underlying principles of the questions asked, and understanding can mean that the interviewee knows the questions to ask.

[0077] In a possible implementation, evaluation prompt words are used to assemble the answer text information, including: based on the evaluation level and preset standards, the evaluation prompt words are assembled with the answer content to generate a text for evaluation.

[0078] Furthermore, the word count of the evaluation result being less than the word count threshold can increase the speed of generating the evaluation video.

[0079] Through the above method, speech recognition technology is used to convert the interviewee's speech content into text, avoiding the time consumption of manual transcription; the large language model can quickly generate evaluation results through automatic analysis of the answer text, greatly improving the evaluation efficiency of the interview process; the large language model can accurately match the interviewee's self-declared ability level to the interviewee's answer, improve the accuracy of the interview, and meet the requirements of personnel recruitment at different stages.

[0080] In one embodiment, the method further comprises:

[0081] Generate an assessment video based on the digital interviewer image and assessment results;

[0082] Detecting interaction information of a target object, and when the interaction information indicates that the target object is not interactive, loopingly playing a first preset video, and when the interaction information indicates that the target object is interactive, loopingly playing a second preset video;

[0083] When the questioning video or the evaluation video is generated, the questioning video or the evaluation video replaces the second preset video.

[0084] In a possible implementation, the first preset interactive video is a video of the virtual interviewer blinking and slightly shaking, and the second preset interactive video is a video of the virtual interviewer nodding and making an affirmative sound;

[0085] When the question video or the evaluation video is generated, a video link of the question video or the evaluation video is determined, and according to the video link, the question video or the evaluation video is used to replace the preset interactive video, and the question video or the evaluation video is played to the target object;

[0086] When no questions are asked or comment videos are played, the first preset interactive video or the second preset interactive video can be played to the target object in a loop to improve the immersion and interactivity during the interview.

[0087] Through the above method, by looping the preset interactive video, the immersion and interactivity in the interview process are improved, the nervousness of the interviewee during the waiting process is reduced, and the interview enthusiasm of the target object is improved in an interactive way.

[0088] In one embodiment, detecting the interaction information of the target object includes:

[0089] Obtain the audio stream of the target object during the interview process, use a filter to remove background noise from the audio stream, and determine whether the audio stream includes the target object's voice signal based on energy threshold detection;

[0090] Obtain the target object's video stream during the interview process, use the face detection algorithm to determine the target object's face area, and determine the changes in the target object's mouth feature points based on lip reading recognition technology;

[0091] The joint analysis results of the audio stream and the video stream are obtained. When the joint analysis results represent that the changes in the mouth feature points match the voice signal at the same time, it is determined that there is interaction between the target objects.

[0092] In a possible implementation, an audio stream of a target object during an interview is obtained, a filter is used to remove background noise of the audio stream, and based on energy threshold detection, it is determined whether the audio stream includes a speech signal of the target object, including:

[0093] Acquire the target subject's audio stream data from the recording device during the interview in real time;

[0094] Use filters or noise reduction algorithms to remove background noise from audio stream data and retain the valid part of the audio signal;

[0095] Energy analysis is performed on the filtered audio signal. When the energy of the signal in the audio stream exceeds a preset threshold, the system determines that a possible speech signal has been detected. The energy threshold can be dynamically adjusted according to the ambient noise conditions to improve the accuracy of speech detection.

[0096] Here, the noise reduction algorithm may include but is not limited to: a frequency domain filtering algorithm and a time domain filtering algorithm.

[0097] In a possible implementation, a joint analysis result of the audio stream and the video stream is obtained, and when the joint analysis result indicates that a change in a mouth feature point matches a voice signal at the same time, it is determined that there is interaction with the target object, including:

[0098] The system timestamp is used to synchronize the audio stream with the video stream. When a voice signal is detected in the audio stream, the system also detects whether the feature points of the mouth in the video stream have changed accordingly.

[0099] Combining other facial features or environmental sounds to further reduce false detections, other facial features include but are not limited to: facial direction, eye expression changes.

[0100] Here, by combining lip reading recognition technology with mouth features, the system can effectively eliminate background noise and other interference, achieve accurate interaction detection, and ensure that the analysis of interactive information during the interview process is more accurate and reliable.

[0101] In one embodiment, the method further comprises:

[0102] Obtain the target object's audio and video streams, use a feature extraction algorithm to extract the target object's facial feature points, determine the target object's emotional type during the interview based on a pre-trained facial emotion recognition model, and use a posture estimation algorithm to detect the target object's body posture during the interview;

[0103] Using voice activity detection technology, the audio stream is divided into voice segments and non-voice segments, the target object's speaking speed is determined based on the number of sentences in the voice segments within a preset time, the interval length between the voice segments and the non-voice segments is obtained, and the pause situation of the target object is determined based on the interval length;

[0104] A multimodal psychological state assessment model is used to determine the target object's comprehensive interview situation as a preset assessment criterion based on at least one of the target object's emotional type, body posture, speech speed and pause conditions.

[0105] In a possible implementation, a feature extraction algorithm is used to extract facial feature points of a target object, and the emotion type of the target object during the interview is determined based on a pre-trained facial emotion recognition model, including:

[0106] Use feature extraction algorithms to obtain key facial feature points of the target object, such as the location information of eyes, eyebrows, nose, mouth, etc.;

[0107] The pre-trained facial emotion recognition model is used to identify the target object’s emotion type based on the extracted facial feature points.

[0108] Here, the pre-trained facial emotion recognition model may include a convolutional neural network based emotion classifier.

[0109] In one possible implementation, a posture estimation algorithm is used to detect the body posture of the target subject during the interview, including: using a posture estimation algorithm to detect the body posture of the target subject, including the angles and positions of the head, arms, shoulders, torso and other parts, and analyzing whether the target subject is in a relaxed or tense state.

[0110] Among them, the pose estimation algorithm may include OpenPose and Posenet.

[0111] In a possible implementation, a voice activity detection technology is used to segment an audio stream into a voice segment and a non-voice segment, a speaking speed of a target object is determined based on the number of sentences within a preset time in the voice segment, an interval length between the voice segment and the non-voice segment is obtained, and a pause condition of the target object is determined based on the interval length, including:

[0112] Based on the segmented speech segments, count the number of sentences of the target object within the preset time window;

[0113] Calculate the target subject’s speaking speed based on the number of sentences per unit time;

[0114] By analyzing the interval between speech segments and non-speech segments, the duration and frequency of the target object's pauses when speaking are calculated to determine whether the target object is nervous, thoughtful or anxious.

[0115] Exemplarily, the preset time window may be 10 seconds.

[0116] Through the above method, a comprehensive evaluation is made on the performance of the target object from multiple dimensions, providing interviewers with richer and more objective information to help them make accurate judgments. The automation of the entire process from interview position matching, automatic question generation to voice and video interviews, intelligent comments and feedback is realized, which can ensure the objectivity and fairness of the interview, help reduce the workload of the human resources department, and improve the quality and speed of talent selection.

[0117] In a specific embodiment, Figure 2 FIG. 1 is a schematic diagram of a specific process of an automatic interview method based on artificial intelligence provided by an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0118] Step S201, looping and playing a first preset video;

[0119] Here, start the interview and loop the first preset video;

[0120] Step S202, determine the position and upload the resume;

[0121] Here, determine the target's resume file and job position;

[0122] Step S203, determining keyword information;

[0123] Here, keyword information is determined based on the resume file and job post;

[0124] Step S204, generating questions;

[0125] Step S205, generating question voice information;

[0126] Here, the question voice information corresponding to each question is determined;

[0127] Step S206, generating a question video;

[0128] Here, based on the digital human generation technology, the question voice information is converted into a question video;

[0129] Step S207, generating a video link;

[0130] Step S208, replacing the preset video;

[0131] Here, the question video replaces the first preset video;

[0132] Step S209, obtaining the target object's answer content;

[0133] Here, the interviewee's answer to the question video is obtained;

[0134] Step S210, speech recognition;

[0135] Step S211, prompt word assembly;

[0136] Step S212, generating an evaluation result;

[0137] Here, the evaluation results are generated based on the answer content;

[0138] Step S213, converting the evaluation result into speech;

[0139] Step S214, generating an evaluation video;

[0140] Here, the scoring speech is converted into an evaluation video based on the digital human generation technology;

[0141] Step S215, generating a video link;

[0142] Step S216, replacing the preset video;

[0143] Here, the first preset video or the second preset video is replaced according to the evaluation video;

[0144] Step S217, determine whether there are more questions, if so, go to step S206, if not, go to step S218;

[0145] Here, determine if there are any more questions to ask;

[0146] Step S218, end.

[0147] In a specific embodiment, Figure 3 is a schematic diagram of questions asked by an automatic interview method based on artificial intelligence provided by an embodiment of the present invention, such as Figure 3 As shown in the figure, a large language model is used to generate three questions for Java engineer positions. Figure 4 This is a schematic diagram of an evaluation result of an automatic interview method based on artificial intelligence provided by an embodiment, obtaining the target object for Figure 3 The answers to the three questions are assembled with prompt words, and the corresponding evaluation results are generated using a large language model.

[0148] Figure 5 is a schematic diagram of the structure of an automatic interview device based on artificial intelligence provided by an embodiment of the present invention, such as Figure 5 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: a questioning module 501, an evaluation module 502 and an interaction module 503;

[0149] The questioning module 501 is used to obtain keyword information for the applied position in the resume file of the target object, and based on the keyword information and the job requirements of the applied position, use a large language model to generate questions for the target object; the keyword information is used to represent the matching degree between the target object and the applied position;

[0150] An evaluation module 502 is used to generate a corresponding question video based on the question, obtain the answer content of the target object to the question video, and use the large language model to determine the evaluation result corresponding to the answer content based on a preset evaluation standard;

[0151] The interaction module 503 is used to generate a corresponding evaluation video based on the evaluation result, so that the target object can obtain the evaluation result.

[0152] In one embodiment, the questioning module 501 is used to obtain a resume file of a target object, and use a large language model to perform keyword extraction processing on the resume file to obtain keyword information in the resume file;

[0153] Determine whether the job position in the keyword information is consistent with the applied position. When the job position is consistent with the applied position, randomly determine prompt word information in a preset prompt word library corresponding to the applied position based on the keyword information, combine the prompt word information with a preset template to obtain a prompt sentence, and based on the prompt sentence, use the large language model to generate the question with a word count less than a word count threshold; the keyword information includes but is not limited to: the job position, project experience, and ability level.

[0154] In one embodiment, the evaluation module 502 is used to generate a corresponding digital interviewer image based on the resume file of the target object using digital human generation technology;

[0155] The text information of at least one question generated by the large language model is obtained, and the voice information of each question corresponding to the question is determined by using text-to-speech technology, and a question video corresponding to each question is generated based on the image of the digital interviewer, the text information and the voice information of the question.

[0156] In one embodiment, the evaluation module 502 is used to obtain the answering voice of the target object by using speech recognition technology, and obtain the answering text information corresponding to the answering voice by using speech-to-text technology;

[0157] Determine a corresponding evaluation level based on the ability level in the keyword information of the target object; the evaluation levels include: first level, second level and third level;

[0158] Based on the preset evaluation criteria and the evaluation level, evaluation prompt words are determined, the answer text information is assembled with prompt words using the evaluation prompt words, and the evaluation result corresponding to the answer content is generated using the large language model, wherein the number of words in the evaluation result is less than a word count threshold.

[0159] In one embodiment, the apparatus further includes: a playing module 504;

[0160] A playback module 504, configured to generate the evaluation video based on the digital interviewer image and the evaluation result;

[0161] Detecting interaction information of the target object, and when the interaction information indicates that the target object is not interactive, loopingly playing a first preset video, and when the interaction information indicates that the target object is interactive, loopingly playing a second preset video;

[0162] When the questioning video or the evaluation video is generated, the questioning video or the evaluation video replaces the second preset video.

[0163] In one embodiment, the playback module 504 is used to obtain the audio stream of the target object during the interview process, use a filter to remove background noise from the audio stream, and determine whether the audio stream includes the voice signal of the target object based on energy threshold detection;

[0164] Obtain a video stream of the target object during the interview process, use a face detection algorithm to determine the face area of ​​the target object, and determine changes in the target object's mouth feature points based on lip reading recognition technology;

[0165] A joint analysis result of the audio stream and the video stream is obtained, and when the joint analysis result indicates that a change in the mouth feature point matches the voice signal at the same time, it is determined that there is interaction with the target object.

[0166] In one embodiment, the apparatus further comprises: an analysis module 505;

[0167] An analysis module 505 is used to obtain an audio stream and a video stream of a target object, extract facial feature points of the target object using a feature extraction algorithm, determine the emotion type of the target object during the interview based on a pre-trained facial emotion recognition model, and detect the body posture of the target object during the interview using a posture estimation algorithm;

[0168] Using voice activity detection technology, the audio stream is divided into voice segments and non-voice segments, the speaking speed of the target object is determined based on the number of sentences in the voice segments within a preset time, the interval length between the voice segments and the non-voice segments is obtained, and the pause situation of the target object is determined based on the interval length;

[0169] A multimodal psychological state assessment model is adopted to determine the comprehensive interview situation of the target object as a preset assessment criterion based on at least one of the target object's emotion type, body posture, speech speed and pause situation.

[0170] It should be noted that: the above-mentioned embodiment provides an automatic interview device based on artificial intelligence, when implementing the corresponding automatic interview method based on artificial intelligence, only the division of the above-mentioned program modules is used as an example. In actual application, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-mentioned processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0171] The embodiment of the present invention also provides a computer device having the above Figure 5 The artificial intelligence-based automatic interview device shown.

[0172] See also Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of another remote data backup device provided by an embodiment of the present invention. Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0173] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0174] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0175] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0176] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0177] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0178] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0179] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0180] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0181] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0182] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. An automatic interview method based on artificial intelligence, characterized in that: The method comprises: Obtain keyword information for the applied position in the resume file of the target object, and generate questions for the target object using a large language model based on the keyword information and the job requirements of the applied position; the keyword information is used to characterize the degree of match between the target object and the applied position; Generate a corresponding question video based on the question, obtain the target object's answer content to the question video, and use the large language model to determine the evaluation result corresponding to the answer content based on preset evaluation criteria; Based on the evaluation result, a corresponding evaluation video is generated so that the target object obtains the evaluation result.

2. The method according to claim 1, characterized in that: The step of obtaining keyword information for the job being applied for in the resume file of the target object, and using a large language model to generate questions for the target object based on the keyword information and the job requirements of the job being applied for, includes: Obtain a resume file of a target object, and use a large language model to perform keyword extraction processing on the resume file to obtain keyword information in the resume file; Determine whether the job position in the keyword information is consistent with the applied position. When the job position is consistent with the applied position, randomly determine prompt word information in a preset prompt word library corresponding to the applied position based on the keyword information, combine the prompt word information with a preset template to obtain a prompt sentence, and based on the prompt sentence, use the large language model to generate the question with a word count less than a word count threshold; the keyword information includes but is not limited to: the job position, project experience, and ability level.

3. The method according to claim 2, characterized in that The generation of the corresponding question video by the question-asking method is achieved based on the following steps: Using digital human generation technology to generate a corresponding digital interviewer image based on the resume file of the target object; The text information of at least one question generated by the large language model is obtained, and the voice information of each question corresponding to the question is determined by using text-to-speech technology, and a question video corresponding to each question is generated based on the image of the digital interviewer, the text information and the voice information of the question.

4. The method according to claim 3, characterized in that: Obtain the answer content of the target object to the question video, and use the large language model to determine the evaluation result corresponding to the answer content based on a preset evaluation standard; the preset evaluation standard is determined based on the keyword information, and is implemented based on the following steps: Using speech recognition technology to obtain the target object's answering speech, and using speech-to-text technology to obtain answering text information corresponding to the answering speech; Determine a corresponding evaluation level based on the ability level in the keyword information of the target object; the evaluation levels include: first level, second level and third level; Based on the preset evaluation criteria and the evaluation level, evaluation prompt words are determined, the answer text information is assembled with prompt words using the evaluation prompt words, and the evaluation result corresponding to the answer content is generated using the large language model, wherein the number of words in the evaluation result is less than a word count threshold.

5. The method according to claim 4, characterized in that The method further comprises: generating the evaluation video based on the digital interviewer image and the evaluation result; Detecting interaction information of the target object, and when the interaction information indicates that the target object is not interactive, loopingly playing a first preset video, and when the interaction information indicates that the target object is interactive, loopingly playing a second preset video; When the questioning video or the evaluation video is generated, the questioning video or the evaluation video replaces the second preset video.

6. The method according to claim 5, characterized in that The detecting the interactive information of the target object includes: Acquire an audio stream of the target object during the interview process, use a filter to remove background noise from the audio stream, and determine whether the audio stream includes a voice signal of the target object based on energy threshold detection; Obtain a video stream of the target object during the interview process, use a face detection algorithm to determine the face area of ​​the target object, and determine changes in the target object's mouth feature points based on lip reading recognition technology; A joint analysis result of the audio stream and the video stream is obtained, and when the joint analysis result indicates that a change in the mouth feature point matches the voice signal at the same time, it is determined that there is interaction with the target object.

7. The method according to claim 6, characterized in that The method further comprises: Acquire the audio stream and video stream of the target object, extract the facial feature points of the target object using a feature extraction algorithm, determine the emotion type of the target object during the interview based on a pre-trained facial emotion recognition model, and detect the body posture of the target object during the interview using a posture estimation algorithm; Using voice activity detection technology, the audio stream is divided into voice segments and non-voice segments, the speaking speed of the target object is determined based on the number of sentences in the voice segments within a preset time, the interval length between the voice segments and the non-voice segments is obtained, and the pause situation of the target object is determined based on the interval length; A multimodal psychological state assessment model is adopted to determine the comprehensive interview situation of the target object as a preset assessment criterion based on at least one of the target object's emotion type, body posture, speech speed and pause situation.

8. An automatic interview device based on artificial intelligence, characterized in that: The device comprises: A questioning module is used to obtain keyword information for the applied position in the resume file of the target object, and based on the keyword information and the job requirements of the applied position, use a large language model to generate questions for the target object; the keyword information is used to represent the matching degree between the target object and the applied position; An evaluation module, configured to generate a corresponding question video based on the question, obtain the target object's answer content to the question video, and use the large language model to determine an evaluation result corresponding to the answer content based on a preset evaluation standard; The interactive module is used to generate a corresponding evaluation video based on the evaluation result so that the target object can obtain the evaluation result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the automatic interview method based on artificial intelligence described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the artificial intelligence-based automatic interview method described in any one of claims 1 to 7.

Images