Interview video analysis method and device, storage medium and product

Abstract

The invention discloses an interview video analysis method and device, a storage medium and a product, and relates to the technical field of video analysis, and the method comprises the steps: obtaining a to-be-analyzed interview video; a preset large model is called to carry out quantitative scoring on interviewer image information and interviewer reply content in the to-be-analyzed interview video, an image quantitative score and a reply quantitative score are obtained, and the large model is adjusted in advance through a preset image anchor point factor and a reply anchor point factor; and according to the image quantitative score and the reply quantitative score, performing analysis to obtain an interview video analysis result. According to the method, the defects of semantic deviation, image misjudgment and rule stiffness in a traditional method can be effectively overcome based on collaborative analysis of a multi-modal large model and a quantitative mechanism guided by the anchor point factors, then a high-credibility interview analysis result is determined, and finally intelligent, standardized and high-reliability enabling of the video interview process is completed.

Description

Technical Field

[0001] This application relates to the field of video analytics technology, and in particular to methods, devices, storage media, and products for analyzing interview videos. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, AI video interview systems have been widely used in recruitment, online education, and remote assessment. These systems typically identify the audio content in interview videos and combine it with image analysis technology to make a preliminary judgment on the interviewee's appearance and behavior, thereby achieving automated scoring and analysis.

[0003] In related technologies, interview assessments are often conducted using traditional machine learning and rule engines. For example, speech recognition models are used to convert interview answers into text, and then keyword matching or shallow semantic analysis methods are relied upon for content evaluation. However, traditional speech recognition models have a high error rate when facing complex environments (such as background noise, accent differences, and speech rate variations). There is a discrepancy between the transcribed text and the true semantics, which directly affects the accuracy of subsequent content analysis, resulting in severely distorted evaluation content and low credibility of the interview video analysis results. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, storage medium and product for analyzing interview videos, aiming to solve the technical problem of low reliability of interview video analysis results.

[0005] To achieve the above objectives, this application proposes an interview video analysis method, which includes:

[0006] Obtain the interview videos to be analyzed;

[0007] A pre-defined large model is invoked to quantify and score the interviewee's image information and response content in the interview video to be analyzed, resulting in an image quantitative score and a response quantitative score. The large model is pre-adjusted by a pre-defined image anchor factor and a response anchor factor.

[0008] Based on the quantitative scores of the image and the quantitative scores of the responses, the results of the interview video analysis are obtained.

[0009] In one embodiment, the step of calling a preset large model to quantitatively score the interviewee's image information and the interviewee's answers in the interview video to be analyzed, and obtaining the image quantitative score and the answer quantitative score, includes:

[0010] Based on a pre-set speech recognition model, speech recognition and semantic calibration are performed on the interview video to be analyzed to obtain the interviewee's response content. Then, an anchor point analysis is performed on the interviewee's response content through a pre-set content evaluation model to obtain the interviewee's question and answer dataset.

[0011] Based on a pre-set image assessment model, anchor point analysis of the interviewee's image is performed on the interview video to be analyzed to obtain an interviewee image analysis dataset.

[0012] Using a pre-defined multimodal large model, the interviewee question and answer dataset and the interviewee image analysis dataset are quantitatively evaluated to determine the quantitative score of the answer and the quantitative score of the image.

[0013] In one embodiment, the step of performing speech recognition and semantic calibration on the interview video to be analyzed based on a preset speech recognition model to obtain the interviewee's response includes:

[0014] Extract the audio signal from the interview video to be analyzed;

[0015] The audio signal is subjected to endpoint detection and segmentation to obtain several audio segments;

[0016] The entire audio segment is transcribed based on the speech recognition model of the large speech recognition model to obtain preliminary transcribed text;

[0017] The initial texts are spliced ​​together and aligned with timestamps to form a complete transcribed text.

[0018] The interview questions and the complete transcribed text are input into the semantic calibration model of the speech recognition big data model. The complete transcribed text is calibrated through contextual semantic analysis, and the calibrated interviewee's response is output.

[0019] In one embodiment, the step of performing anchor point analysis on the interviewee's responses using a preset content evaluation model to obtain the interviewee's question-answer dataset includes:

[0020] If the interview scenario corresponding to the interviewee's answer is an open-ended question interview scenario, then the anchor points of the preset content evaluation model are configured according to the preset open-ended anchor point evaluation system. The preset open-ended anchor point evaluation system includes four assessment directions: appropriate word choice, rich content, communication expression, and logical thinking.

[0021] By configuring the content evaluation model, anchor point matching analysis is performed on the interviewee's answers to obtain quantitative scores for each anchor point. The quantitative scores of the anchor points are then compiled to obtain the interviewee's question and answer dataset.

[0022] In one embodiment, the step of performing anchor point analysis on the interviewee's responses using a preset content evaluation model to obtain the interviewee's question-answer dataset includes:

[0023] If the interview scenario corresponding to the interviewee's answer is a semi-structured question interview scenario, construct a semi-structured anchor point evaluation system that includes twelve core competency dimensions: career stability, learning ability, initiative, leadership, service awareness, stress resistance, teamwork, innovation ability, results orientation, efficient execution, relationship building, and conscientiousness. Based on the semi-structured anchor point evaluation system, configure the content evaluation big model.

[0024] The step of performing anchor point analysis on the interviewee's responses using a pre-defined content evaluation model to obtain the interviewee's question-and-response dataset includes:

[0025] The interviewee's answers, the corresponding interview questions, and the core competency dimensions are input into the content evaluation model, and each anchor factor is quantitatively scored through anchor matching analysis.

[0026] The scores of each anchor factor are aggregated to generate the interviewee question and answer dataset.

[0027] In one embodiment, the step of performing anchor point analysis on the interviewee's image in the interview video to be analyzed based on a preset image assessment model to obtain an interviewee image analysis dataset includes:

[0028] The interview video to be analyzed is subjected to frame extraction processing to obtain multiple frames of images;

[0029] Based on the image anchor point assessment system that includes four assessment points: appearance, clothing, hairstyle, and environment, a multimodal large model is configured;

[0030] The multi-frame images are input into the multimodal large model, and each anchor factor is quantitatively scored through anchor point matching analysis.

[0031] The scores of each anchor factor are aggregated to generate the interviewee image analysis dataset.

[0032] In one embodiment, the step of analyzing the interview video analysis results based on the image quantification score and the response quantification score includes the following prior steps:

[0033] The interview video to be analyzed is sampled frame by frame to obtain a sampled frame sequence;

[0034] The face detection model is used to perform face recognition on each of the sampled frames to determine whether there are abnormal situations such as multiple faces, faces outside the frame, or occlusion.

[0035] An eye-tracking model is used to detect the direction of eye movements in the identified facial regions to determine whether there are any abnormal behaviors such as eye movement deviation.

[0036] Based on preset anomaly detection rules, whether to output an abnormal behavior identifier depends on whether there is an abnormal scenario in the face recognition result and whether there is abnormal behavior in the eye direction detection result.

[0037] In addition, to achieve the above objectives, this application also proposes an interview video analysis device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the interview video analysis method as described above.

[0038] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the interview video analysis method described above.

[0039] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the interview video analysis method described above.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] Compared to related technologies, which suffer from high error rates in complex environments (such as background noise, accent differences, and speech rate variations), and discrepancies between transcribed text and true semantics, directly impacting the accuracy of subsequent content analysis and resulting in severely distorted evaluation content, leading to low credibility of interview video analysis results, this application addresses these issues by acquiring the interview video to be analyzed; calling a pre-set large model to quantify and score the interviewee's image information and responses in the interview video, obtaining an image quantification score and a response quantification score. The large model is pre-adjusted using pre-set image anchor factors and response anchor factors; and analyzing the interview video based on the image quantification score and the response quantification score to obtain the interview video analysis results. Understandably, this application employs a technical architecture combining multi-module collaboration, a multimodal large-scale model, and a refined anchor point evaluation system. When processing interview videos to be analyzed, the speech recognition and large-scale model semantic calibration module effectively suppresses environmental interference and speech-to-text errors, achieving accurate restoration of the response content. Furthermore, the structured anchor point system enables fine-grained decoupling and quantitative evaluation of both content and image dimensions, thereby achieving a deep, objective, and stable digital assessment of the interviewee's overall performance. Therefore, the collaborative analysis based on the multimodal large-scale model and the quantitative mechanism guided by anchor point factors can effectively overcome the shortcomings of traditional methods, such as semantic bias, image misjudgment, and rigid rules, thus determining highly reliable interview analysis results and ultimately empowering the video interview process with intelligence, standardization, and high reliability. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating an embodiment of the interview video analysis method of this application.

[0045] Figure 2 This is a flowchart illustrating Embodiment 2 of the interview video analysis method of this application;

[0046] Figure 3 This is a flowchart illustrating Embodiment 3 of the interview video analysis method of this application;

[0047] Figure 4 This is a flowchart illustrating Embodiment 4 of the interview video analysis method of this application;

[0048] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the interview video analysis method in this application embodiment.

[0049] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0051] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution in this application's embodiments is:

[0053] Obtain the interview videos to be analyzed;

[0054] A pre-defined large model is invoked to quantify and score the interviewee's image information and response content in the interview video to be analyzed, resulting in an image quantitative score and a response quantitative score. The large model is pre-adjusted by a pre-defined image anchor factor and a response anchor factor.

[0055] Based on the quantitative scores of the image and the quantitative scores of the responses, the results of the interview video analysis are obtained.

[0056] In this embodiment, the application uses an interview video analysis device as the execution subject. For ease of description, it will be referred to as "device" in the following detailed description.

[0057] Because existing technologies have a high error rate when facing complex environments (such as background noise, accent differences, and changes in speech rate), there are discrepancies between the transcribed text and the true semantics. This directly affects the accuracy of subsequent content analysis, resulting in severely distorted evaluation content and low credibility of the interview video analysis results.

[0058] This application provides a solution that employs a technical architecture combining multi-module collaboration, a multimodal large-scale model, and a refined anchor point evaluation system. When processing interview videos to be analyzed, the speech recognition and large-scale model semantic calibration module effectively suppresses environmental interference and speech-to-text errors, achieving accurate restoration of the response content. Furthermore, the structured anchor point system performs fine-grained decoupling and quantitative evaluation of both content and image dimensions, thereby achieving a deep, objective, and stable digital assessment of the interviewee's overall performance. Therefore, the collaborative analysis based on the multimodal large-scale model and the quantitative mechanism guided by anchor point factors can effectively overcome the shortcomings of traditional methods, such as semantic bias, image misjudgment, and rigid rules, thus determining highly reliable interview analysis results and ultimately enabling the intelligent, standardized, and highly reliable video interview process.

[0059] Based on this, embodiments of this application provide a method for analyzing interview videos, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the interview video analysis method of this application.

[0060] In this embodiment, the interview video analysis method includes steps S10 to S30:

[0061] Step S10: Obtain the interview video to be analyzed;

[0062] It should be noted that the interview video to be analyzed refers to a complete video record obtained through interview recording equipment, containing information such as the interviewee's voice, image, and behavior, for subsequent quantitative analysis. It can be a series of video clips recorded continuously during the interview, or a collection of multiple clips.

[0063] Step S20: Call the preset large model to quantify and score the interviewee's image information and the interviewee's answers in the interview video to be analyzed, and obtain the image quantitative score and the answer quantitative score. The large model is pre-adjusted by preset image anchor factors and answer anchor factors.

[0064] It should be noted that the preset large model is a deep learning model that has been trained and optimized, including but not limited to Qwen2.5-32B, Qwen2VL, etc., which is specifically designed to process the voice and image data in interview videos to extract key features and perform evaluation.

[0065] The interviewee's image information includes their physical characteristics, including but not limited to facial features, clothing, hairstyle, and environment. This information is analyzed using a multimodal large-scale model to quantitatively evaluate the interviewee's image performance. The interviewee's response content refers to their answers to questions. This is transcribed into text using a speech recognition model, and then semantically understood and logically analyzed using a large-scale model to quantitatively evaluate the quality of the response. The quantitative score is based on preset anchor factors, numerically evaluating the interviewee's image and response content to generate specific scores that visually reflect their performance. The image quantitative score is specifically a quantitative result of the interviewee's image information, reflecting their performance in terms of image. The response quantitative score is specifically a quantitative result of the interviewee's response content, reflecting their comprehensive performance in terms of content logic, word accuracy, etc. Anchor factors are preset evaluation indicators used to guide the large-scale model to conduct detailed analysis on specific dimensions, such as logic, innovativeness, and facial neatness, and serve as the basis for the quantitative score.

[0066] Understandably, by invoking a pre-set large model and using anchor factors for pre-adjustment, a comprehensive and objective quantitative assessment of the interviewee's image and responses can be achieved. This assessment method not only captures deep semantic and image features that are difficult to identify using traditional methods, but also effectively reduces the subjectivity of manual assessment, improves the accuracy and reliability of interview video analysis, and provides strong support for recruitment, education assessment, and other scenarios.

[0067] For example, refer to Figure 2 Imagine an online job interview where the system acquires a candidate's interview video. First, the system uses a pre-defined model to transcribe and semantically calibrate the audio portion of the video, ensuring the accuracy of the responses. Next, based on anchor factors such as "logic" and "innovation," the system quantifies and scores the responses, resulting in a quantitative response score. Simultaneously, the system samples video frames to analyze the candidate's facial features, clothing, and other physical characteristics, quantifying these features based on anchor factors such as "groominess" and "appropriateness of attire," resulting in a quantitative image score. Finally, the system integrates these quantitative scores to generate a comprehensive interview analysis report, helping recruiters quickly and objectively evaluate candidates.

[0068] In one feasible implementation, the step of calling a preset large model to quantitatively score the interviewee's image information and the interviewee's answers in the interview video to be analyzed, and obtaining the image quantitative score and the answer quantitative score, includes:

[0069] Based on a pre-set speech recognition model, speech recognition and semantic calibration are performed on the interview video to be analyzed to obtain the interviewee's response content. Then, an anchor point analysis is performed on the interviewee's response content through a pre-set content evaluation model to obtain the interviewee's question and answer dataset.

[0070] Based on a pre-set image assessment model, anchor point analysis of the interviewee's image is performed on the interview video to be analyzed to obtain an interviewee image analysis dataset.

[0071] Using a pre-defined multimodal large model, the interviewee question and answer dataset and the interviewee image analysis dataset are quantitatively evaluated to determine the quantitative score of the answer and the quantitative score of the image.

[0072] It should be noted that the pre-set large models are trained and optimized deep learning models specifically designed to process the audio and image data from interview videos. For example, Qwen2.5-32B is used for speech and semantic processing, while the Qwen2VL series is used for multimodal image assessment. Pre-set large models for speech recognition, such as SenseVoice Small, can extract audio signals from videos and transcribe them into text. Semantic calibration utilizes these large models to perform contextual semantic analysis and correction on the transcribed text, improving accuracy. Pre-set large models for content assessment, like Qwen2.5-32B, can perform anchor point analysis on responses, extract key information, and quantify the assessment. The interviewee question-and-answer dataset is a collection of text content and assessment results after anchor point analysis. Pre-set large models for image assessment, such as Qwen2VL-72B-Int4, are responsible for performing anchor point analysis on the interviewee's image and generating image assessment results. The interviewee image analysis dataset is a collection of analyzed image information and assessment results. The pre-defined multimodal large model, such as Qwen2VL-7B, can process text and image data simultaneously, perform quantitative evaluation on the response dataset and the image dataset, and finally determine the quantitative score of the response and the quantitative score of the image.

[0073] Understandably, this implementation method, through multi-module collaboration, achieves a comprehensive quantitative assessment of interviewees' responses and appearance, effectively improving the accuracy and reliability of interview video analysis and providing strong support for recruitment decisions.

[0074] In one feasible implementation, the step of performing speech recognition and semantic calibration on the interview video to be analyzed based on a preset speech recognition model to obtain the interviewee's response includes:

[0075] Extract the audio signal from the interview video to be analyzed;

[0076] The audio signal is subjected to endpoint detection and segmentation to obtain several audio segments;

[0077] The entire audio segment is transcribed based on the speech recognition model of the large speech recognition model to obtain preliminary transcribed text;

[0078] The initial texts are spliced ​​together and aligned with timestamps to form a complete transcribed text.

[0079] The interview questions and the complete transcribed text are input into the semantic calibration model of the speech recognition big data model. The complete transcribed text is calibrated through contextual semantic analysis, and the calibrated interviewee's response is output.

[0080] It should be noted that audio signals refer to pure sound data extracted from video files, which contains the interviewee's voice information and is the basic input for speech recognition processing.

[0081] Endpoint detection and segmentation refers to using technical means to identify active speech segments in audio, remove silent or background noise parts, and segment continuous audio into shorter segments suitable for speech recognition, thereby improving recognition efficiency and accuracy.

[0082] Preliminary transcribed text refers to the initial text result generated after the speech recognition model automatically transcribes audio segments. It may contain certain recognition errors, such as word misrecognition due to accents or background noise.

[0083] Splicing and timestamp alignment refers to merging multiple preliminary transcribed text fragments in the original audio order and marking the corresponding time point for each text fragment to ensure the consistency between the transcribed content and the video process, which facilitates subsequent analysis.

[0084] Semantic calibration models are large-scale language models trained using deep learning techniques. They can understand the contextual semantic relationships of natural language, perform semantic understanding and correction on the initial transcribed text, correct semantic deviations caused by speech recognition errors, and make the text more consistent with the actual expressive intent.

[0085] Understandably, this step, through the synergy of speech recognition and semantic calibration, effectively enhances the ability to extract accurate textual information from interview videos. The speech recognition model converts audio signals into text, while semantic calibration further ensures the accuracy and semantic coherence of the text, providing a reliable foundation for subsequent text-based in-depth analysis (such as logical and professional assessments), reducing the chain of analytical biases caused by speech recognition errors, and enhancing the robustness and credibility of the entire interview assessment system.

[0086] For example, referring to Table 1, in an online interview for a campus recruitment position, a student applying for a data analysis position experienced errors in the speech recognition system due to poor network signal and his own accent. When asked, "Please talk about how you use data analysis techniques to solve practical problems," he answered, "During my sophomore summer vacation, I attended a data mining course at Harvard University." However, the speech recognition model incorrectly transcribed this as: "In my sophomore years, during my summer vacation, I went to Harvard for a summer school course in data mining for business." Subsequently, the semantic calibration model intervened, analyzing the contextual semantics and the keyword "data mining for business" to correct the text to the accurate expression: "In my sophomore year, during the summer vacation, I went to Harvard for a summer school course in data mining for business."

[0087] In another example, the interviewer asked, "Please talk about your childhood dream and how it influenced your career choice." The interviewee replied, "When I was a child, I dreamed of becoming a scientist so that I could change the world." However, due to unclear speech, the speech recognition model incorrectly transcribed it as: "My dreams have been changed. O child? I dreamed to be a Sentenceist so that I can change the world." The semantic calibration model again came into play, correcting the text to the correct expression based on the contextual semantics and the keywords "scientist" and "change the world": "My dreams have been changed. When I was a child, I dreamed to be a scientist so that I can change the world." Through this semantic calibration process, the accuracy of the interview system was significantly improved, providing recruiters with a more reliable evaluation basis.

[0088] Table 1. Examples of Large Model Calibration Comparison

[0089]

[0090] In one feasible implementation, the step of performing anchor point analysis on the interviewee's responses using a preset content evaluation model to obtain the interviewee's question-and-answer dataset includes the following:

[0091] If the interview scenario corresponding to the interviewee's answer is an open-ended question interview scenario, then the anchor points of the preset content evaluation model are configured according to the preset open-ended anchor point evaluation system. The preset open-ended anchor point evaluation system includes four assessment directions: appropriate word choice, rich content, communication expression, and logical thinking.

[0092] By configuring the content evaluation model, anchor point matching analysis is performed on the interviewee's answers to obtain quantitative scores for each anchor point. The quantitative scores of the anchor points are then compiled to obtain the interviewee's question and answer dataset.

[0093] It's important to note that the open-ended anchor point assessment system is a set of evaluation indicators specifically designed for open-ended interview scenarios. It encompasses four main assessment areas: appropriate word choice, rich content, communication and expression, and logical thinking. Each area is further refined into multiple specific anchor point factors to comprehensively evaluate the quality of the interviewee's responses. Appropriate word choice assesses whether the interviewee used accurate and appropriate vocabulary, including anchor point factors such as accurate word choice, common conjunctions, excellent conjunctions, correct tense, correct voice, and advanced vocabulary. Rich content assesses the depth and breadth of the interviewee's response, including anchor point factors such as relevance of content, sufficient and comprehensive evidence, and clear arguments. Communication and expression analyze the interviewee's fluency, naturalness of expression, and emotional delivery, covering anchor point factors such as fluency, completeness of answer, initiative, values, natural expression, emotional stability, and quick thinking. Logical thinking assesses the coherence, coherence, and logic of the interviewee's response, including anchor point factors such as coherence, coherence, reasonableness, clear viewpoints, broad thinking, and illustrative examples. Anchor point matching analysis refers to the process by which a large-scale content assessment model extracts and matches features from interviewees' responses based on pre-defined anchor point factors to quantify the score for each anchor point. The anchor point quantification score is a specific score obtained after quantifying each anchor factor, reflecting the interviewee's performance in the corresponding assessment area. The interviewee question-and-answer dataset is a collection of data formed by integrating all anchor point quantification scores, comprehensively recording the interviewees' performance in open-ended question scenarios.

[0094] Understandably, by working in tandem with the open-ended anchor assessment system and the content assessment model, this step can comprehensively and deeply quantify and evaluate an interviewee's performance in open-ended question scenarios. It overcomes the limitations of traditional methods that rely solely on keyword matching or superficial semantic analysis, accurately capturing deeper characteristics of responses such as logic, innovation, and language organization skills. This provides recruiters with more detailed and multi-dimensional assessment criteria, enhancing the professionalism and reliability of interview evaluations.

[0095] For example, referring to Table 2, in an open-ended interview for a marketing position, the interviewer asks, "Please share an experience of how you successfully resolved a team conflict." After the interviewee answers, the system initiates anchor point analysis based on a pre-defined content assessment model. The model evaluates the response item by item according to the four assessment directions and their detailed anchor point factors of the open-ended anchor point assessment system. For example, in the "appropriate word choice" direction, the model identifies that the interviewee used advanced English vocabulary such as "coordination" and "win-win," while in the "logical thinking" direction, the model captures that the interviewee's Chinese response is clear and logical, progressing step by step from the cause of the conflict, the resolution process to the final result, with sufficient argumentation. After anchor point matching analysis, the system generates a quantitative score for each anchor point and integrates them to form a dataset of interviewee question and answer responses. Finally, recruiters can use this dataset to intuitively understand the interviewee's overall performance in open-ended question scenarios, helping to determine whether they meet the job requirements.

[0096] Table 2 Anchor Factors for Open-ended Questions

[0097]

[0098] In one feasible implementation, the step of performing anchor point analysis on the interviewee's responses using a preset content evaluation model to obtain the interviewee's question-and-answer dataset includes the following:

[0099] If the interview scenario corresponding to the interviewee's answer is a semi-structured question interview scenario, construct a semi-structured anchor point evaluation system that includes twelve core competency dimensions: career stability, learning ability, initiative, leadership, service awareness, stress resistance, teamwork, innovation ability, results orientation, efficient execution, relationship building, and conscientiousness. Based on the semi-structured anchor point evaluation system, configure the content evaluation big model.

[0100] The step of performing anchor point analysis on the interviewee's responses using a pre-defined content evaluation model to obtain the interviewee's question-and-response dataset includes:

[0101] The interviewee's answers, the corresponding interview questions, and the core competency dimensions are input into the content evaluation model, and each anchor factor is quantitatively scored through anchor matching analysis.

[0102] The scores of each anchor factor are aggregated to generate the interviewee question and answer dataset.

[0103] It should be noted that semi-structured interview scenarios refer to interview questions with a certain fixed framework, but allowing interviewees to answer freely. These scenarios are usually used to assess specific ability dimensions of interviewees.

[0104] The twelve core competency dimensions refer to career stability, learning ability, initiative, leadership, service awareness, resilience, teamwork, innovation, results orientation, efficient execution, relationship building, and conscientiousness. These dimensions are the comprehensive qualities that companies focus on when interviewing candidates.

[0105] The semi-structured anchor point assessment system is an assessment system designed for semi-structured interview scenarios. It includes the above twelve core competency dimensions, each with a specific anchor point factor, used to accurately assess the interviewee's performance in each competency dimension.

[0106] Anchor factors refer to specific evaluation indicators under each core competency dimension, used to quantify the interviewee's performance in that dimension, such as "excellent job stability" or "good learning ability".

[0107] Understandably, this step, by constructing a semi-structured anchor point assessment system and utilizing a large-scale content assessment model to quantitatively analyze interviewees' responses, can accurately evaluate their performance across twelve core competency dimensions. This assessment method overcomes the problems of single assessment indicators and strong subjectivity in traditional methods, achieving a comprehensive and objective quantitative assessment of interviewees' overall abilities and providing a scientific basis for recruitment decisions.

[0108] For example, referring to Table 3, in a semi-structured interview for a management position, the interviewer asks, "Please share an experience of successfully completing a project under high pressure." The interviewee's answer, along with the corresponding interview question and twelve core competency dimensions, is input into the content assessment model. The model performs anchor point matching analysis on the answers based on the semi-structured anchor point assessment system. For instance, in the "stress resistance" dimension, the model identifies statements mentioned by the interviewee such as "efficiently adjusting work plans before deadlines" and "remaining calm and motivating the team," matching them with corresponding anchor point factors and quantifying their scores. Finally, the system summarizes the scores of each anchor point factor, generating a dataset of interviewee answers, providing recruiters with intuitive and comprehensive assessment results to help determine whether the interviewee possesses the core competencies required for the position.

[0109] Table 3 Anchor Factors for Semi-structured Problems

[0110] Key points of the inspection Anchor factor Job stability Excellent job stability, good job stability Learning ability Excellent learning ability, good learning ability be proactive Striving for excellence, striving for good leadership Excellent leadership, good leadership service awareness Excellent leadership, good leadership Pressure resistance Excellent and good compressive strength Teamwork Excellent teamwork, good teamwork Innovation capability Excellent innovation ability, good innovation ability Results-oriented Excellent results-oriented, good results-oriented High-efficiency execution Excellent high-efficiency execution, good high-efficiency execution Relationship establishment Excellent relationship building, good relationship building Conscientious and responsible Conscientious and responsible, excellent; conscientious and responsible, good

[0111] In one feasible implementation, the step of performing anchor point analysis on the interviewee's image in the interview video to be analyzed, based on a preset image assessment model, to obtain the interviewee image analysis dataset includes:

[0112] The interview video to be analyzed is subjected to frame extraction processing to obtain multiple frames of images;

[0113] Based on the image anchor point assessment system that includes four assessment points: appearance, clothing, hairstyle, and environment, a multimodal large model is configured;

[0114] The multi-frame images are input into the multimodal large model, and each anchor factor is quantitatively scored through anchor point matching analysis.

[0115] The scores of each anchor factor are aggregated to generate the interviewee image analysis dataset.

[0116] It should be noted that frame extraction refers to extracting image frames from a video at certain time intervals to generate a set of static image data for subsequent image analysis.

[0117] The image anchor point assessment system refers to an assessment system that takes appearance, clothing, hairstyle and environment as the four major assessment points. Each key point contains several specific anchor point factors, which are used to comprehensively and quantitatively assess the interviewee's image performance.

[0118] Multimodal large models refer to deep learning models that can process multiple types of data (such as images and text), such as Qwen2VL-72B-Int4, which is used for anchor point analysis and quantitative evaluation of interviewees' image.

[0119] Anchor point matching analysis refers to the process by which a multimodal large model extracts and matches features from input image data based on preset anchor point factors, generating a specific score for each anchor point.

[0120] Quantitative scoring involves numerically evaluating each anchor factor to generate a specific score, reflecting the interviewee's performance in the corresponding image dimension.

[0121] The interviewee image analysis dataset is a collection of data formed by integrating all anchor point quantitative scores, comprehensively recording the interviewee's performance in terms of image.

[0122] Understandably, by using frame extraction and anchor point analysis of a multimodal large model, this step can comprehensively and objectively quantify and evaluate the interviewee's image performance. This method overcomes the limitations of traditional image assessments that rely on subjective judgment and standardized video training. It can maintain high generalization ability and assessment accuracy in unstructured scenarios such as different lighting and shooting angles, providing recruiters with intuitive and reliable interviewee image assessment results.

[0123] For example, referring to Table 4, in a video interview for an account manager position, the system extracts multiple frames from the interview video, covering different expressions and postures of the interviewee. Based on the image anchor point evaluation system, a multimodal large model analyzes these images. For example, at the "face" anchor point, the model identifies that the interviewee's expression is natural, their eyes are focused, their facial features are symmetrical, and there are no exaggerated accessories, matching the corresponding anchor point and quantifying a score of 4.2 (out of 5). At the "clothing" anchor point, the model detects that the interviewee is dressed formally, their shirt color is appropriate, and their clothing fits well, quantifying a score of 4.8. At the "hairstyle" anchor point, the model analyzes that the hairstyle is neat and tidy, without any exaggerated elements like hair dye, scoring 4.5. At the "environment" anchor point, the model identifies that the background is clean and free of clutter, scoring 4.3. Finally, the system summarizes the scores of each anchor point, generating an interviewee image analysis dataset, providing recruiters with a comprehensive image evaluation basis to help determine whether the interviewee meets the image requirements of the position.

[0124] Table 4 Image Assessment Anchor Factors

[0125]

[0126]

[0127] Step S30: Based on the image quantification score and the response quantification score, the interview video analysis results are obtained.

[0128] It should be noted that the interview video analysis results refer to a comprehensive evaluation conclusion obtained by integrating the quantitative scores of the interviewee's image and responses, used to reflect the interviewee's overall performance in the current interview scenario. This result generates an intuitive evaluation by combining the relative scores of the interviewee's image dimension and the content of their responses with anchor factors and quantitative evaluation rules.

[0129] Understandably, this step enables a comprehensive and objective evaluation of interviewee performance, effectively integrating quantitative information from the two key dimensions of appearance and responses, overcoming the problems of strong subjectivity and insufficient quantification in traditional interview assessments. Through precise quantitative analysis, it provides reliable data support for recruitment decisions, improving the scientific rigor and efficiency of interview assessments.

[0130] In one feasible implementation, the step of analyzing the interview video analysis results based on the image quantification score and the response quantification score includes the following prior steps:

[0131] The interview video to be analyzed is sampled frame by frame to obtain a sampled frame sequence;

[0132] The face detection model is used to perform face recognition on each of the sampled frames to determine whether there are abnormal situations such as multiple faces, faces outside the frame, or occlusion.

[0133] An eye-tracking model is used to detect the direction of eye movements in the identified facial regions to determine whether there are any abnormal behaviors such as eye movement deviation.

[0134] Based on preset anomaly detection rules, whether to output an abnormal behavior identifier depends on whether there is an abnormal scenario in the face recognition result and whether there is abnormal behavior in the eye direction detection result.

[0135] It should be noted that frame sampling refers to extracting image frames from the interview video at preset intervals (such as sampling once every 15 frames) to generate a set of sampled frame sequences for subsequent face detection and eye tracking analysis.

[0136] A sampled frame sequence refers to a series of image frames obtained after frame sampling, arranged in chronological order, used to represent key moments in a video.

[0137] Face detection models are models based on computer vision technology, such as RetinaFace, used to identify the location, size, and number of faces in an image, and to determine whether there are abnormal situations such as multiple faces, faces outside the frame, or faces being occluded.

[0138] Eye-tracking models refer to deep learning-based eye-tracking technology models, such as L2CS-Net, which are used to detect the direction of eye movements in the face region in real time and determine whether there are abnormal eye movement behaviors.

[0139] Eye direction detection refers to analyzing the direction and fixation point of the eyes using an eye-tracking model to determine whether the gaze deviates from the normal range.

[0140] Anomaly detection rules refer to pre-defined logical rules that determine whether an abnormal behavior flag is triggered based on the results of face recognition and eye direction detection. For example, an abnormal face scene is identified when the proportion of multiple face frames exceeds 5% or the proportion of valid face frames is less than 95%; an abnormal eye behavior is identified when the proportion of frames with eye deviation in the same direction exceeds 15%.

[0141] Abnormal behavior identifiers are labels or indications used to mark whether cheating or other abnormal behavior exists in an interview video.

[0142] Understandably, by working together frame sampling, face detection, eye tracking, and anomaly detection rules, this step can effectively identify and flag abnormal behavior before analyzing the interview video, providing a clean and reliable data foundation for subsequent quantitative evaluation of appearance and responses. This not only improves the credibility of the evaluation results but also enhances the system's cheating detection capabilities, ensuring the fairness and impartiality of the interview process.

[0143] For example, refer to Figure 3In a video interview for a technical position, the system first samples the interview video frame by frame, extracting one frame every 15 frames to generate a sampled frame sequence. The face detection model then analyzes these sampled frames, finding that 5% of the frames contain multiple faces (e.g., other people appearing next to the interviewee). The eye-tracking model further detects that the interviewee exhibits prolonged eye-shifting behavior at certain times, with frames showing shifts exceeding 15 degrees accounting for 20%. Based on pre-defined anomaly detection rules, the system determines that the interview video is suspected of cheating and outputs an abnormal behavior flag. Upon receiving this flag, recruiters can further review the relevant video clips to confirm whether any violations have occurred, thus ensuring the fairness of the interview and the validity of the results.

[0144] This embodiment provides a method for analyzing interview videos. The application employs a technical architecture combining multi-module collaboration, a multimodal large-scale model, and a refined anchor point evaluation system. When processing the interview video to be analyzed, the speech recognition and large-scale model semantic calibration module effectively suppresses environmental interference and speech-to-text errors, achieving accurate restoration of the response content. Furthermore, the structured anchor point system is used to perform fine-grained decoupling and quantitative evaluation of the content and image dimensions, thereby achieving a deep, objective, and stable digital assessment of the interviewee's overall performance. Therefore, the collaborative analysis based on the multimodal large-scale model and the anchor point factor-guided quantitative mechanism can effectively overcome the shortcomings of traditional methods, such as semantic bias, image misjudgment, and rigid rules, thus determining highly reliable interview analysis results and ultimately enabling the intelligent, standardized, and highly reliable video interview process.

[0145] For example, to help understand the implementation process of the interview video analysis method obtained by combining this embodiment with the first embodiment described above, please refer to... Figure 4 , Figure 4 A simplified flowchart of an interview video analysis method is provided, specifically:

[0146] Step 1: Obtain the interview videos to be analyzed

[0147] Obtain a complete video file containing the interviewee's voice, image, and behavioral information from the interview recording device or storage medium.

[0148] Step 2: Speech Recognition and Semantic Calibration

[0149] The audio signal in the video is transcribed using a pre-set speech recognition model to obtain preliminary text; then, the accuracy of the text is optimized by combining the semantic calibration model with the contextual semantics.

[0150] Step 3: Content Evaluation

[0151] Based on the type of open-ended or semi-structured interview questions, a pre-defined content assessment model is invoked. Using an anchor point assessment system, the interviewee's responses are quantitatively scored, generating a question-response dataset. For open-ended questions, the anchor point system covers four assessment areas: appropriate word choice, rich content, communication skills, and logical thinking. For semi-structured questions, the anchor point system focuses on twelve core competency dimensions.

[0152] Step Four: Image Assessment

[0153] Using a pre-defined image assessment model, anchor point analysis is performed on the video images after frame extraction. The image is assessed from four key aspects: face, clothing, hairstyle, and environment. The results are quantified, scored, and an image analysis dataset is generated.

[0154] Step 5: Abnormal Behavior Detection

[0155] Sampling of video frames yields a sequence of sampled frames.

[0156] The face detection model is used to identify multiple faces, faces outside the frame, or faces that are occluded.

[0157] An eye-tracking model is used to detect eye direction and determine eye-shifting behavior.

[0158] Based on preset standards, if the proportion of multiple face frames exceeds 5% or the proportion of eye-shift frames exceeds 15%, it is determined whether to output an abnormal behavior flag.

[0159] Step Six: Comprehensive Analysis of Output Results

[0160] By integrating quantitative scores of appearance, quantitative scores of responses, and abnormal behavior detection results, a comprehensive interview video analysis report is generated.

[0161] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the interview video analysis method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0162] This application provides an interview video analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the interview video analysis method in Embodiment 1 above.

[0163] The following is for reference. Figure 5The diagram illustrates a structural schematic of an interview video analysis device suitable for implementing embodiments of this application. The interview video analysis device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 5 The interview video analysis device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0164] like Figure 5 As shown, the interview video analysis device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the interview video analysis device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the interview video analysis device to communicate wirelessly or wiredly with other devices to exchange data. While the figures show interview video analysis devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0165] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0166] The interview video analysis device provided in this application, employing the interview video analysis method in the above embodiments, can solve the technical problem of low reliability of interview video analysis results. Compared with the prior art, the beneficial effects of the interview video analysis device provided in this application are the same as those of the interview video analysis method provided in the above embodiments, and other technical features in this interview video analysis device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0167] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0169] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the interview video analysis method in the above embodiments.

[0170] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0171] The aforementioned computer-readable storage medium may be included in the interview video analysis device; or it may exist independently and not assembled into the interview video analysis device.

[0172] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the interview video analysis device, cause the interview video analysis device to: acquire the interview video to be analyzed;

[0173] A pre-defined large model is invoked to quantify and score the interviewee's image information and response content in the interview video to be analyzed, resulting in an image quantitative score and a response quantitative score. The large model is pre-adjusted by a pre-defined image anchor factor and a response anchor factor.

[0174] Based on the quantitative scores of the image and the quantitative scores of the responses, the results of the interview video analysis are obtained.

[0175] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0177] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0178] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described interview video analysis method, thereby solving the technical problem of low reliability of interview video analysis results. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the interview video analysis method provided in the above embodiments, and will not be repeated here.

[0179] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the interview video analysis method described above.

[0180] The computer program product provided in this application can solve the technical problem of low reliability of interview video analysis results. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the interview video analysis method provided in the above embodiments, and will not be repeated here.

[0181] All acquisition of signals, information, or actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the relevant device owner.

[0182] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. An interview video analysis method characterized by, The method comprises: acquiring an interview video to be analyzed; calling a preset large model to quantitatively score the interviewee image information and the interviewee reply content in the interview video to be analyzed, to obtain an image quantitative score and a reply quantitative score, wherein the large model is pre-adjusted by preset image anchor point factors and reply anchor point factors; analyzing to obtain an interview video analysis result according to the image quantitative score and the reply quantitative score.

2. The method of claim 1, wherein, The step of calling the preset large model to quantitatively score the interviewee image information and the interviewee reply content in the interview video to be analyzed, to obtain an image quantitative score and a reply quantitative score comprises: based on a preset voice recognition large model, performing voice recognition and semantic calibration on the interview video to be analyzed to obtain the interviewee reply content, and then performing anchor point analysis on the interviewee reply content by a preset content evaluation large model to obtain an interviewee question reply data set; based on a preset image evaluation large model, performing anchor point analysis on the interviewee image in the interview video to be analyzed to obtain an interviewee image analysis data set; performing quantitative evaluation on the interviewee question reply data set and the interviewee image analysis data set respectively by a preset multi-modal large model to determine the reply quantitative score and the image quantitative score.

3. The method of claim 2, wherein, The step of calling the preset voice recognition large model to quantitatively score the interviewee image information and the interviewee reply content in the interview video to be analyzed, to obtain an image quantitative score and a reply quantitative score comprises: extracting an audio signal in the interview video to be analyzed; performing endpoint detection and segmentation processing on the audio signal to obtain a plurality of audio segments; transcribing all the audio segments according to a voice recognition model of the voice recognition large model to obtain preliminary transcription texts; splicing and time stamp aligning each of the preliminary texts to form a complete transcription text; inputting an interview question and the complete transcription text into a semantic calibration model of the voice recognition large model, and calibrating the complete transcription text by context semantic analysis to output calibrated interviewee reply content.

4. The method of claim 2, wherein, The step of calling the preset voice recognition large model to quantitatively score the interviewee image information and the interviewee reply content in the interview video to be analyzed, to obtain an image quantitative score and a reply quantitative score comprises: if the interview scene corresponding to the interviewee reply content is an open question interview scene, then according to a preset open anchor point evaluation system, performing anchor point configuration on the preset content evaluation large model, wherein the open anchor point evaluation system comprises four investigation directions of appropriate word use, content richness, communication expression and logical thinking; performing anchor point matching analysis on the interviewee reply content by the configured content evaluation large model to obtain a plurality of anchor point quantitative scores, and arranging the anchor point quantitative scores to obtain an interviewee question reply data set.

5. The method of claim 2, wherein, The step of calling the preset voice recognition large model to quantitatively score the interviewee image information and the interviewee reply content in the interview video to be analyzed, to obtain an image quantitative score and a reply quantitative score comprises: If the interviewee's answer corresponds to a semi-structured interview scene, a semi-structured anchor point evaluation system including twelve core competence dimensions of career stability, learning ability, initiative, leadership, service consciousness, stress resistance, teamwork, innovation ability, result orientation, efficient execution, relationship establishment, and responsibility is constructed, and the content evaluation large model is configured based on the semi-structured anchor point evaluation system; The step of performing anchor point analysis on the interviewee's answer by the preset content evaluation large model to obtain an interviewee question answer data set comprises: The interviewee's answer, the corresponding interview question, and the core competence dimension are input into the content evaluation large model, and each anchor point factor is quantitatively scored through anchor point matching analysis; The scores of each anchor point factor are summarized to generate the interviewee question answer data set.

6. The method of claim 2, wherein, The step of performing anchor point analysis on the interviewee's image based on the preset image evaluation large model to obtain an interviewee image analysis data set comprises: Frame extraction processing is performed on the interview video to be analyzed to obtain multiple frames of images; A multi-modal large model is configured according to an image anchor point evaluation system including four investigation points of face, clothing, hairstyle, and environment; The multiple frames of images are input into the multi-modal large model, and each anchor point factor is quantitatively scored through anchor point matching analysis; The scores of each anchor point factor are summarized to generate the interviewee image analysis data set.

7. The method of claim 1, wherein, The step of analyzing the interview video analysis result according to the image quantitative score and the answer quantitative score comprises: Frame sampling is performed on the interview video to be analyzed to obtain a sampling frame sequence; A face detection model is used to perform face recognition on each sampling frame to determine whether there is an abnormal situation of multiple faces, face out of frame, or occlusion; An eye tracking model is used to perform eye direction detection on the recognized face area to determine whether there is an abnormal behavior of eye deviation; Based on a preset abnormality judgment rule, whether to output an abnormal behavior identifier is determined according to whether the face recognition result has an abnormal situation and whether the eye direction detection result has an abnormal behavior.

8. An interview video analysis apparatus characterized by comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the interview video analysis method according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the interview video analysis method according to any one of claims 1 to 7.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the steps of the interview video analysis method according to any one of claims 1 to 7.

Images