Interview simulation analysis system based on deep learning

Abstract

The invention relates to the technical field of deep learning of artificial intelligence, in particular to an interview simulation analysis system based on deep learning, and the system comprises a data collection module which collects the comprehensive performance feature information of a standard interviewer in a target interview simulation system; the data analysis module analyzes the performance characteristic representation value based on the comprehensive performance characteristic information so as to analyze the comprehensive performance characteristic index; a standard evaluation module judges whether the comprehensive performance characteristic index is abnormal or not, and preliminarily judges whether the target interview simulation system meets the standard or not based on the total abnormal rate; the erroneous judgment correction module analyzes an outlier data correction total abnormal rate based on the abnormal comprehensive performance characteristic index, and secondarily judges whether the target simulation system meets the standard or not; and the attribution analysis module judges whether the multi-modal deep learning model is abnormal or not based on the abnormal comprehensive performance characteristic index, and generates an evaluation result. According to the method, the evaluation precision of the interview simulation system is remarkably improved, so that the reliability of the interview simulation system is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning of artificial intelligence, and particularly relates to an interview simulation analysis system based on deep learning. BACKGROUND

[0002] With the continuous improvement of the accuracy requirement of enterprises on talent selection, the reliability and accuracy of the interview simulation system are facing higher challenges. The multi-modal model provided by the prior art can extract the performance feature parameters of the interviewee, and provides a preliminary data basis. However, the prior art only stays in the parameter extraction and preliminary scoring stage, lacks a scientific evaluation mechanism for the overall performance of the system, cannot solve the misjudgment problem caused by the state fluctuation of individual interviewees, and cannot accurately locate the fault module when the system is abnormal, which seriously restricts the iterative optimization and practical application value of the interview simulation system.

[0003] Chinese patent publication No. CN117974081A discloses an AI large model-based simulated interview teaching method and system. The present application provides an AI large model-based simulated interview teaching method and system. The method comprises the following steps: obtaining interview data and facial image of a user to be interviewed; obtaining an expression description text from a preset expression open text pool, and inputting the facial image and the expression description text into a multi-modal expression recognition sub-model to obtain a portrait feature representation vector and a text feature representation vector output by the multi-modal expression recognition sub-model; performing similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the facial image based on the similarity result; inputting the education data and the resume data into an AI large model to obtain an education score value and a resume score value output by the AI large model; and determining a simulated interview teaching result of the user to be interviewed based on the expression recognition result, the education score value and the resume score value. The present application improves the accuracy of the simulated interview result.

[0004] Chinese patent publication No. CN118411138A discloses an AI intelligent interview simulation empowerment cabin. The present application provides an AI intelligent interview simulation empowerment cabin, which relates to the field of artificial intelligence. The AI intelligent interview simulation empowerment cabin combines a BERT model and a CNN model to deeply analyze the resume information and facial expressions of job seekers through an AI intelligent interview system, dynamically generates interview questions through a MySQL database, realizes real-time interaction with the job seekers, gives targeted feedback by analyzing the interview performance of the job seekers, and helps the job seekers adapt to the interview to find suitable jobs.

[0005] The Chinese patent publication No. CN119941206A discloses a method, system, device and medium for simulating an interview scene based on a large model, which belongs to the technical field of large models and artificial intelligence, and specifically relates to a method, system, device and medium for simulating an interview scene based on a large model. The method comprises the following steps: obtaining input resume information through a visual interface; preprocessing the input resume information, performing semantic analysis on the input resume by using a large model, extracting key information of the resume in combination with recruitment information of a user company; generating corresponding interview questions based on the analysis result and the key information of the resume, and recording the generated related parameters in real time; until the set number of questions and answers is completed, calculating an interview evaluation index based on the reply information of the job seeker and the resume information; comparing the interview evaluation index with a preset threshold range to obtain an evaluation scheme corresponding to the interview evaluation index; and presenting the generated questions, the corresponding job seeker reply information and the evaluation scheme to the user in a visual form. When facing a job seeker, the interview success rate can be improved.

[0006] Therefore, the prior art has the following problems: The existing interview simulation system is not considered for analysis and evaluation, resulting in low reliability of the interview simulation system. SUMMARY

[0007] To this end, the present application provides an interview simulation analysis system based on deep learning to overcome the problem of low reliability of the interview simulation system caused by not considering the analysis and evaluation of the existing interview simulation system in the prior art.

[0008] To achieve the above-mentioned purpose, the present application provides an interview simulation analysis system based on deep learning, comprising: A data acquisition module is used to acquire comprehensive performance characteristic information of a plurality of standard interviewers in a target interview simulation system; wherein the comprehensive performance characteristic information includes expression performance characteristic parameters, text performance characteristic parameters and interactive performance characteristic parameters; A data analysis module is connected with the data acquisition module, and analyzes corresponding performance characteristic representation values based on the expression performance characteristic parameters, the text performance characteristic parameters and the interactive performance characteristic parameters of each standard interviewer; and analyzes a comprehensive performance characteristic index based on the performance characteristic representation values of each standard interviewer; A standard evaluation module is connected with the data analysis module, and compares the comprehensive performance characteristic index of each standard interviewer with a preset comprehensive performance characteristic index threshold to determine whether the comprehensive performance characteristic index is abnormal; and preliminarily determines whether the target interview simulation system meets the standard based on the overall abnormal rate of a plurality of standard interviewers; a misjudgment correction module connected with the data collection module and the standard evaluation module, based on the comprehensive performance characteristic index of the abnormality, analyzes the outlier data of the corresponding comprehensive performance characteristic information, corrects the overall abnormality rate based on the outlier data, and determines whether the target simulation system meets the standard based on the corrected overall abnormality rate; an attribution analysis module connected with the data analysis module and the misjudgment correction module, based on the performance characteristic representation value corresponding to the comprehensive performance characteristic index of the abnormality, determines whether the multi-modal deep learning model in the target interview simulation system is abnormal, and generates an evaluation result; The expression performance characteristic parameters include: portrait-text feature vector similarity and emotional state stability of the standard interviewee; the text performance characteristic parameters include: reply accuracy score and resume keyword matching degree; and the interaction performance characteristic parameters include: interview question response time and multi-round question and answer coherence.

[0009] Further, the data analysis module determines the expression performance characteristic representation value of each standard interviewee based on the first expression performance characteristic parameter and the second expression performance characteristic parameter, determines the text performance characteristic representation value of each standard interviewee based on the first text performance characteristic parameter and the second text performance characteristic parameter, and determines the interaction performance characteristic representation value of each standard interviewee based on the first interaction performance characteristic parameter and the second interaction performance characteristic parameter. The first expression performance characteristic parameter and the second expression performance characteristic parameter are calculated by the portrait-text feature vector similarity and the emotional state stability of each standard interviewee, respectively. The first text performance characteristic parameter and the second text performance characteristic parameter are calculated by the reply accuracy score and the resume keyword matching degree of each standard interviewee, respectively. The first interaction performance characteristic parameter and the second interaction performance characteristic parameter are calculated by the interview question response time and the multi-round question and answer coherence of each standard interviewee, respectively.

[0010] Further, the data analysis module determines the comprehensive performance characteristic index of each standard interviewee based on the weighted sum of the first comprehensive performance characteristic parameter, the second comprehensive performance characteristic parameter, and the third comprehensive performance characteristic parameter. The first comprehensive performance characteristic parameter, the second comprehensive performance characteristic parameter, and the third comprehensive performance characteristic parameter are calculated by the expression performance characteristic representation value, the text performance characteristic representation value, and the interaction performance characteristic representation value of each standard interviewee, respectively.

[0011] Further, the standard evaluation module is used to determine that the comprehensive standard characteristic index is abnormal when the comprehensive performance characteristic index is less than a preset comprehensive performance characteristic index threshold.

[0012] Furthermore, the standard evaluation module preliminarily determines that the target interview simulation system does not meet the standard based on the fact that the overall abnormality rate of several standard interviewees is greater than a preset abnormality rate threshold.

[0013] Furthermore, the misjudgment correction module acquires comprehensive performance feature information corresponding to several abnormal comprehensive performance feature indices, and extracts outlier data from the comprehensive performance feature information using the interquartile range method.

[0014] Furthermore, the misjudgment correction module is used to remove the comprehensive performance characteristic index corresponding to the outlier data from a number of abnormal comprehensive performance characteristic indices, thereby correcting the overall abnormality rate of a number of standard interviewees.

[0015] Furthermore, the misjudgment correction module determines that the target interview simulation system does not meet the standard based on the fact that the overall abnormality rate after correction is greater than the preset abnormality rate threshold.

[0016] Furthermore, the attribution analysis module determines whether the multimodal deep learning model in the target interview simulation system is abnormal based on the mean of the performance characteristic representation values ​​corresponding to the comprehensive performance characteristic index of several anomalies after the elimination is completed.

[0017] Furthermore, if the mean value of the facial expression feature representation is less than a preset threshold value, the attribution analysis module determines that there is an anomaly in the facial expression analysis of the multimodal deep learning model; if the mean value of the text expression feature representation is less than a preset threshold value, the attribution analysis module determines that there is an anomaly in the text understanding and evaluation of the multimodal deep learning model; and if the mean value of the interaction expression feature representation is less than a preset threshold value, the interaction evaluation of the multimodal deep learning model determines that there is an anomaly.

[0018] Compared with existing technologies, the beneficial effects of this invention are that it provides a deep learning-based interview simulation analysis system. By constructing a complete system architecture including data acquisition, data analysis, standard evaluation, misjudgment correction, and attribution analysis, this invention achieves a multi-dimensional comprehensive evaluation of the performance of the interview simulation system. The data acquisition module systematically acquires three major categories of feature parameters: facial expressions, text, and interaction, providing a comprehensive data foundation for evaluation. The data analysis module transforms the raw parameters into standardized representation values, eliminating the influence of dimensions and making data from different dimensions comparable. The standard evaluation module achieves preliminary anomaly detection through exponential threshold comparison, establishing quantitative evaluation standards. The misjudgment correction module effectively eliminates false alarms caused by abnormal states of individual interviewees through outlier detection, significantly improving the robustness of the evaluation results. The attribution analysis module can accurately locate the source of system failure, providing a clear direction for subsequent optimization. This invention, through the collaborative cooperation of multiple modules to evaluate and analyze the interview simulation system, significantly improves the evaluation accuracy of the interview simulation system, thereby improving the reliability and accuracy of the interview simulation system.

[0019] In particular, by using the data analysis module to calculate the performance characteristic representation values ​​for the three dimensions of expression, text, and interaction based on the subdivided parameters of performance characteristic information, a unified dimensional transformation of parameters of different types and magnitudes is achieved, eliminating analytical biases caused by differences in parameter units and making the subsequent comprehensive index calculation more comparable and reasonable. The correspondence between each representation value and the specific original parameters is clearly defined, and each calculation step has a clear data source, ensuring the traceability and reproducibility of the evaluation process and further improving the credibility of the representation values. This design makes the integrated analysis of multi-dimensional data more standardized and accurate, laying a solid foundation for the reliable calculation of the subsequent comprehensive performance characteristic index and improving the standardization of data processing and the accuracy of evaluation results. By introducing a weighted summation mechanism to calculate the comprehensive performance characteristic index, adaptive evaluation for the interview requirements of different positions is achieved. The first, second, and third comprehensive performance characteristic parameters correspond to the three dimensions of expression, text, and interaction, respectively, and the weights can be adjusted to flexibly adapt to the evaluation focus of different positions. This dynamic weighting mechanism makes the evaluation results more targeted and practical, accurately reflecting the performance of the interview simulation system in different application scenarios and providing precise guidance for system optimization.

[0020] In particular, by setting a threshold for the comprehensive performance characteristic index in the standard evaluation module and comparing it, an objective and unified standard for anomaly judgment was established. When the comprehensive performance characteristic index is less than this threshold, it is judged as an anomaly. This quantitative judgment method ensures the objectivity and consistency of the evaluation results. Simultaneously, the threshold comparison mechanism responds quickly, is suitable for real-time evaluation scenarios, and can promptly identify problematic systems, preventing the negative impact of continuing to use unqualified systems. By introducing the concept of overall anomaly rate and setting a threshold for the anomaly rate, a quantitative evaluation of the overall performance of the interview simulation system was achieved. The overall anomaly rate comprehensively reflects the system's performance across all test samples. When the overall anomaly rate exceeds the preset threshold, the system is initially judged to be non-compliant with the standard. This statistically based judgment method greatly improves the reliability and credibility of the evaluation results.

[0021] In particular, the interquartile range (ICM) method, employed in the misjudgment correction module, extracts outlier data, providing a statistically sound and computationally efficient method for misjudgment identification. The ICM is not stringent regarding data distribution, making it applicable to feature data of various distribution types and exhibiting excellent robustness. This method can accurately identify outliers that differ significantly from the rest of the data. These outliers often stem from individual interviewees' performance rather than system malfunctions, providing a reliable basis for subsequent misjudgment correction.

[0022] In particular, by removing outlier indices and recalculating the anomalous rate, the evaluation results achieve self-correction and optimization. This correction mechanism effectively filters out false alarms caused by individual interviewees' abnormal states, making the evaluation results more accurately reflect the system's performance. The corrected overall anomalous rate focuses more on evaluating common system problems, improving the accuracy and reliability of the evaluation results. This dynamic correction mechanism gives the system a certain degree of fault tolerance, avoiding incorrect judgments based on individual anomalous samples, and significantly enhancing the practical value and reliability of the evaluation system.

[0023] In particular, a dual-verification evaluation mechanism was established based on a secondary judgment using the corrected overall anomaly rate, further ensuring the accuracy of the evaluation results. The initial judgment mainly involves rapid screening, while the secondary judgment makes a precise assessment after eliminating interfering factors. This dual-verification mechanism significantly reduces the probability of misjudgment, making the final conclusion more scientific and credible. When the corrected anomaly rate is still greater than the threshold, it indicates that there is a general problem in the system rather than an isolated phenomenon, requiring timely repair and optimization. This rigorous judgment standard ensures that only truly qualified systems can pass the evaluation, maintaining the authority and effectiveness of the evaluation system.

[0024] In particular, by analyzing the mean of the performance characteristic representation values ​​corresponding to the anomaly index through the attribution analysis module, fault diagnosis and performance evaluation of the multimodal deep learning model were achieved. This mean-based evaluation method can reflect the overall performance level of the module, focusing more on common problems existing in the system itself and improving the accuracy of diagnosis. By setting thresholds for representation values ​​of different dimensions and comparing them separately, precise module-by-module localization of model anomalies was achieved, directly locating problems in expression analysis, text understanding, or interactive evaluation sub-models. This evaluation method provides a clear target direction for system optimization, helping developers quickly locate problematic modules, significantly improving maintenance efficiency, making system optimization more targeted, and accelerating the iterative cycle of system performance improvement. Attached Figure Description

[0025] Figure 1 This is a structural block diagram of a deep learning-based interview simulation analysis system according to an embodiment of the present invention; Figure 2 The present invention provides a logical determination diagram for identifying anomalies in the comprehensive standard feature index. Figure 3 The present invention provides a preliminary logic diagram for determining whether the target interview simulation system does not meet the standards. Figure 4 The logical decision diagram for determining whether the target interview simulation system does not meet the standards in the secondary determination of the present invention. Detailed Implementation

[0026] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0027] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0028] Please see Figure 1 , Figure 2 , Figure 3 as well as Figure 4 The diagram shown is a structural block diagram of a deep learning-based interview simulation analysis system according to an embodiment of the present invention; a logical decision diagram for determining an anomaly in the comprehensive standard feature index according to an embodiment of the present invention; a logical decision diagram for initially determining that the target interview simulation system does not meet the standard according to an embodiment of the present invention; and a logical decision diagram for secondarily determining that the target interview simulation system does not meet the standard according to an embodiment of the present invention.

[0029] This invention provides a deep learning-based interview simulation analysis system, comprising: The data acquisition module is used to collect comprehensive performance characteristic information of several standard interviewees in the target interview simulation system; wherein, the comprehensive performance characteristic information includes facial expression characteristic parameters, text expression characteristic parameters, and interaction expression characteristic parameters. The data analysis module, which is connected to the data acquisition module, analyzes the corresponding performance characteristic values ​​based on the facial expression characteristic parameters, text expression characteristic parameters, and interaction performance characteristic parameters of each standard interviewee; and analyzes the comprehensive performance characteristic index based on the performance characteristic values ​​of each standard interviewee. The standard evaluation module, which is connected to the data analysis module, determines whether the comprehensive performance characteristic index of each standard interviewee is abnormal by comparing it with a preset comprehensive performance characteristic index threshold; and preliminarily determines whether the target interview simulation system meets the standard based on the overall abnormality rate of several standard interviewees. The misjudgment correction module, which is connected to the data acquisition module and the standard evaluation module, analyzes outlier data corresponding to comprehensive performance characteristic information based on several abnormal comprehensive performance characteristic indices; corrects the overall anomalous rate based on the outlier data; and makes a secondary determination on whether the target simulation system meets the standard based on the corrected overall anomalous rate. The attribution analysis module, which is connected to the data analysis module and the misjudgment correction module, determines whether the multimodal deep learning model in the target interview simulation system is abnormal based on the performance characteristic representation values ​​corresponding to several abnormal comprehensive performance characteristic indices, and generates evaluation results.

[0030] The facial expression feature parameters include the similarity between the standard interviewee's image and text feature vectors and the stability of their emotional state; the text expression feature parameters include the accuracy score of the response and the keyword matching degree of the resume; and the interaction expression feature parameters include the response time of the interview questions and the coherence of multiple rounds of question and answer.

[0031] It is understood that the comprehensive performance feature information is obtained through analysis using existing multimodal deep learning models. Specifically, the image-text feature vector similarity is calculated using the multimodal expression recognition sub-model in existing technology CN117974081A, which calculates the cosine similarity value of the image-text feature vectors; the emotional state stability is calculated using the CNN model in existing technology CN118411138A, which calculates the Euclidean distance between the feature vectors of consecutive face frames; the response accuracy score is evaluated using the BERT model in existing technology CN118411138A, which assesses the fluency of the response text; the resume keyword matching degree is calculated using the TF-IDF algorithm and BERT model in existing technology CN119941206A, which calculates the keyword overlap rate between the resume and the job description; the question response time is determined by recording the time interval between the end of the question and the start of the response; and the multi-turn question-and-answer coherence is based on the contextual attention mechanism of the BERT model, extracting global semantic features from multiple turns of dialogue, calculating the correlation between text features in adjacent turns, and outputting a coherence score.

[0032] By constructing a complete system architecture encompassing data acquisition, data analysis, standard evaluation, misjudgment correction, and attribution analysis, this invention achieves a multi-dimensional comprehensive evaluation of the performance of an interview simulation system. The data acquisition module systematically acquires three major categories of feature parameters: facial expressions, text, and interaction, providing a comprehensive data foundation for the evaluation. The data analysis module transforms raw parameters into standardized representation values, eliminating the influence of dimensions and making data from different dimensions comparable. The standard evaluation module achieves preliminary anomaly detection through exponential threshold comparison, establishing quantitative evaluation standards. The misjudgment correction module effectively eliminates false alarms caused by abnormal states of individual interviewees through outlier detection, significantly improving the robustness of the evaluation results. The attribution analysis module can accurately locate the source of system failures, providing a clear direction for subsequent optimization. This invention, through the collaborative work of multiple modules to evaluate and analyze the interview simulation system, significantly improves the evaluation accuracy of the interview simulation system, thereby enhancing its reliability and accuracy.

[0033] Specifically, the data analysis module determines the facial expression feature values ​​of each standard interviewee based on the first facial expression feature parameter and the second facial expression feature parameter; determines the text expression feature values ​​of each standard interviewee based on the first text expression feature parameter and the second text expression feature parameter; and determines the interaction expression feature values ​​of each standard interviewee based on the first interaction expression feature parameter and the second interaction expression feature parameter. The first facial expression feature parameter and the second facial expression feature parameter are respectively calculated by the similarity of the portrait-text feature vectors of each standard interviewee and the stability of the emotional state. The first text performance feature parameter and the second text performance feature parameter are calculated using the accuracy scores of each standard interviewee's responses and the keyword matching degree of the resume, respectively. The first interaction performance feature parameter and the second interaction performance feature parameter are calculated by the response time of each standard interviewee to the interview questions and the continuity of multiple rounds of question and answer.

[0034] In this embodiment, the process of calculating the facial expression characteristic representation value of each standard interviewee based on the facial expression characteristic parameters of each standard interviewee includes: The ratio of the similarity between the image and text feature vectors of each standard interviewee to a preset similarity threshold is calculated and determined as the first facial expression feature parameter. The ratio of the emotional state stability of each standard interviewee to the preset emotional state stability threshold is calculated and determined as the second facial expression characteristic parameter. The sum of the first facial expression feature parameter and the second facial expression feature parameter is determined as the facial expression feature representation value.

[0035] In this embodiment, the process of calculating the text performance feature representation value of each standard interviewee based on the text performance feature parameters of each standard interviewee includes: The ratio of the accuracy score of each standard interviewee's response to the preset accuracy score threshold is calculated and determined as the first text performance feature parameter. The ratio of the keyword matching degree of each standard interviewee's resume to the preset keyword matching degree threshold is calculated and determined as the second text performance feature parameter. The sum of the first text performance feature parameter and the second text performance feature parameter is determined as the text performance feature representation value.

[0036] In this embodiment, the process of calculating the interaction performance characteristic representation value of each standard interviewee based on the interaction performance characteristic parameters of each standard interviewee includes: The ratio of the response time of each standard interviewee to the preset response time threshold for interview questions is calculated and determined as the first interaction performance characteristic parameter. The ratio of the multi-round question-and-answer coherence of each standard interviewee to the preset multi-round question-and-answer coherence threshold is calculated and determined as the second interaction performance characteristic parameter. The sum of the first interaction performance feature parameter and the second interaction performance feature parameter is determined as the interaction performance feature representation value.

[0037] In this embodiment, the thresholds corresponding to the above performance characteristic parameters are all obtained in advance and are set as the average of the performance characteristic parameters of each standard interviewee when the interview simulation system meets the standard within the historical period.

[0038] By calculating the performance characteristic representation values ​​for facial expressions, text, and interaction across three dimensions using subdivided parameters of performance feature information, a unified dimensional transformation of parameters of different types and magnitudes is achieved. This eliminates analytical biases caused by differences in parameter units, making subsequent comprehensive index calculations more comparable and reasonable. The correspondence between each representation value and the specific original parameters is clearly defined, and each calculation step has a clear data source, ensuring the assessment process is traceable and reproducible, further enhancing the credibility of the representation values. This design makes the integrated analysis of multi-dimensional data more standardized and accurate, laying a solid foundation for the reliable calculation of the subsequent comprehensive performance characteristic index and improving the standardization of data processing and the accuracy of assessment results.

[0039] Specifically, the data analysis module determines the comprehensive performance characteristic index of each standard interviewee by weighted summation of the first comprehensive performance characteristic parameter, the second comprehensive performance characteristic parameter, and the third comprehensive performance characteristic parameter; The first comprehensive performance feature parameter, the second comprehensive performance feature parameter, and the third comprehensive performance feature parameter are calculated using the facial expression feature value, text expression feature value, and interaction performance feature value of each standard interviewee, respectively.

[0040] In this embodiment, the process of calculating the comprehensive performance characteristic index of each standard interviewee based on the performance characteristic representation values ​​of each standard interviewee includes: The ratio of the facial expression characteristic representation value of each standard interviewee to the preset facial expression characteristic representation value threshold is calculated and determined as the first comprehensive performance characteristic parameter; The ratio of the text performance feature representation value of each standard interviewee to the preset text performance feature representation value threshold is calculated and determined as the second comprehensive performance feature parameter; The ratio of the interaction performance characteristic representation value of each standard interviewee to the preset interaction performance characteristic representation value threshold is determined as the third comprehensive performance characteristic parameter. The first comprehensive performance characteristic parameter, the second comprehensive performance characteristic parameter, and the third comprehensive performance characteristic parameter are weighted and summed to determine the comprehensive performance characteristic index of each standard interviewee.

[0041] In this embodiment, the aforementioned performance characteristic threshold values ​​are all obtained in advance and are set as the average performance characteristic values ​​of each standard interviewee when the interview simulation system meets the standard within the historical period.

[0042] In this embodiment, the weights of the first comprehensive performance characteristic parameter (corresponding to facial expression), the second comprehensive performance characteristic parameter (corresponding to text expression), and the third comprehensive performance characteristic parameter (corresponding to interaction expression) are not fixed but adaptively adjusted according to the type of the target position. Preferably, for technical R&D positions emphasizing accuracy in interview responses, the weight configuration is: facial expression weight 0.2, text weight 0.6, and interaction weight 0.2. For sales and marketing positions emphasizing approachability, the weight configuration can be: facial expression weight 0.4, text weight 0.3, and interaction weight 0.3. For customer service positions emphasizing emotional stability and approachability, the weight configuration is: facial expression weight 0.5, text weight 0.3, and interaction weight 0.2. This dynamic weighting mechanism, tied to the position, makes the evaluation of the comprehensive performance characteristic index more targeted and accurate.

[0043] By introducing a weighted summation mechanism to calculate the comprehensive performance characteristic index, adaptive assessment of the interview requirements for different positions is achieved. The first, second, and third comprehensive performance characteristic parameters correspond to the three dimensions of facial expressions, text, and interaction, respectively. Adjusting the weights allows for flexible adaptation to the assessment focus of different positions. This dynamic weighting mechanism makes the assessment results more targeted and practical, accurately reflecting the performance of the interview simulation system in different application scenarios and providing precise guidance for system optimization.

[0044] Specifically, the standard evaluation module is used to determine that the comprehensive standard characteristic index is abnormal when the comprehensive performance characteristic index is less than a preset comprehensive performance characteristic index threshold.

[0045] In this embodiment, the comprehensive performance characteristic index threshold is obtained in advance and is set as the average of the comprehensive performance characteristic indices of each standard interviewee when the interview simulation system meets the standard within a historical period.

[0046] By setting a threshold for the comprehensive performance characteristic index and comparing it with other systems, an objective and unified standard for anomaly detection was established. When the comprehensive performance characteristic index is below this threshold, the system is considered abnormal. This quantitative method ensures the objectivity and consistency of the evaluation results. Furthermore, the threshold comparison mechanism is fast-responding, suitable for real-time evaluation scenarios, and can promptly identify problematic systems, preventing the negative impact of continuing to use unqualified systems.

[0047] Specifically, the standard evaluation module initially determines that the target interview simulation system does not meet the standard based on the fact that the overall abnormality rate of a number of standard interviewees is greater than a preset abnormality rate threshold.

[0048] In this embodiment, the anomaly rate threshold is obtained in advance and is set as the average of the overall anomaly rate of standard interviewees when the interview simulation system meets the standards within a historical period.

[0049] By introducing the concept of overall anomaly rate and a preset anomaly rate threshold, a quantitative evaluation of the overall performance of the interview simulation system is achieved. The overall anomaly rate comprehensively reflects the system's performance across all test samples. When the overall anomaly rate exceeds the preset threshold, the system is initially judged to be non-compliant with standards. This statistically based judgment method greatly improves the reliability and credibility of the evaluation results. The anomaly rate threshold is set based on historical qualified data, ensuring both the rigor of the evaluation standards and considering the fault tolerance requirements in practical applications. This system-level evaluation mechanism can effectively identify systems with performance degradation or design flaws, providing a scientific basis for system maintenance and updates.

[0050] Specifically, the misjudgment correction module acquires comprehensive performance feature information corresponding to several abnormal comprehensive performance feature indices, and extracts outlier data from the comprehensive performance feature information using the interquartile range method.

[0051] In this embodiment, several sets of data are obtained, corresponding to the comprehensive performance characteristic indices of anomalous interviewees: similarity between their facial and textual feature vectors, stability of their emotional state, accuracy of their responses, keyword matching in their resumes, response time to interview questions, and coherence in multi-round question-and-answer sessions. The interquartile range (ICM) method is used to determine the normal range boundaries of each data set. Data exceeding the normal range boundaries in any data set are identified as outliers. The calculation process for determining the normal range boundaries of each data set using the ICM is existing technology and will not be described further.

[0052] This paper presents a statistically sound and computationally efficient method for identifying outliers by employing the interquartile range (ICM) method. The ICM is not demanding on data distribution and is applicable to feature data of various distribution types, exhibiting excellent robustness. This method accurately identifies outliers that differ significantly from the rest of the data; these outliers often stem from individual interviewees' performance rather than system malfunctions. By performing outlier detection on separate feature parameter data sets, it ensures comprehensive coverage of all possible anomalies. This statistically based outlier detection method is unaffected by subjective factors, providing objective and reliable results and offering an accurate basis for subsequent misjudgment correction.

[0053] Specifically, the misjudgment correction module is used to remove the comprehensive performance characteristic index corresponding to the outlier data from a number of abnormal comprehensive performance characteristic indices, thereby correcting the overall abnormality rate of a number of standard interviewees.

[0054] By removing outlier indices and recalculating the anomaly rate, the evaluation results achieve self-correction and optimization. This correction mechanism effectively filters out false alarms caused by individual interviewees' abnormal states, making the evaluation results more accurately reflect the system's performance. The corrected overall anomaly rate focuses more on evaluating common system problems, improving the accuracy and reliability of the evaluation results. This dynamic correction mechanism gives the system a certain degree of fault tolerance, avoiding incorrect judgments based on individual outliers, and significantly enhancing the practical value and reliability of the evaluation system.

[0055] Specifically, the misjudgment correction module determines that the target interview simulation system does not meet the standard based on the overall abnormality rate after correction being greater than a preset abnormality rate threshold.

[0056] A dual-verification evaluation mechanism was established, using a secondary assessment based on the corrected overall anomaly rate, to further ensure the accuracy of the evaluation results. The initial assessment primarily involves rapid screening, while the secondary assessment performs a precise judgment after eliminating interfering factors. This dual-verification mechanism significantly reduces the probability of misjudgment, making the final conclusion more scientific and credible. When the corrected anomaly rate still exceeds the threshold, it indicates a widespread problem rather than an isolated phenomenon, requiring timely repair and optimization. This rigorous judgment standard ensures that only truly qualified systems pass the evaluation, maintaining the authority and effectiveness of the evaluation system.

[0057] Specifically, the attribution analysis module determines whether the multimodal deep learning model in the target interview simulation system is abnormal based on the mean of the performance characteristic representation values ​​corresponding to the comprehensive performance characteristic index of several anomalies after the elimination is completed.

[0058] By analyzing the mean of the performance characteristic values ​​corresponding to the anomaly index, fault diagnosis and performance evaluation of a multimodal deep learning model were achieved. This mean-based evaluation method can reflect the overall performance level of the modules, focuses more on common problems existing in the system itself, and improves the accuracy of diagnosis.

[0059] Specifically, the attribution analysis module determines that there is an anomaly in the facial expression analysis of the multimodal deep learning model when the mean value of the facial expression feature representation is less than a preset threshold value; it determines that there is an anomaly in the text understanding and evaluation of the multimodal deep learning model when the mean value of the text expression feature representation is less than a preset threshold value; and it determines that there is an anomaly in the interaction evaluation of the multimodal deep learning model when the mean value of the interaction expression feature representation is less than a preset threshold value.

[0060] By setting threshold values ​​for different dimensions and comparing them separately, precise modular localization of model anomalies is achieved. This allows for direct identification of problems in sub-models such as facial expression analysis, text understanding, or interactive evaluation. This evaluation method provides a clear target direction for system optimization, helps developers quickly locate problematic modules, significantly improves maintenance efficiency, makes system optimization more targeted, and accelerates the iterative cycle of system performance improvement.

[0061] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A deep learning-based interview simulation analysis system, characterized in that, include: The data acquisition module is used to collect comprehensive performance characteristic information of several standard interviewees in the target interview simulation system; wherein, the comprehensive performance characteristic information includes: facial expression characteristic parameters, text expression characteristic parameters, and interaction expression characteristic parameters; The data analysis module, which is connected to the data acquisition module, analyzes the corresponding performance characteristic values ​​based on the facial expression characteristic parameters, text expression characteristic parameters, and interaction performance characteristic parameters of each standard interviewee; and analyzes the comprehensive performance characteristic index based on the performance characteristic values ​​of each standard interviewee. The standard evaluation module, which is connected to the data analysis module, determines whether the comprehensive performance characteristic index of each standard interviewee is abnormal by comparing it with a preset comprehensive performance characteristic index threshold; and preliminarily determines whether the target interview simulation system meets the standard based on the overall abnormality rate of several standard interviewees. The misjudgment correction module, which is connected to the data acquisition module and the standard evaluation module, analyzes outlier data corresponding to comprehensive performance characteristic information based on several abnormal comprehensive performance characteristic indices; corrects the overall anomalous rate based on the outlier data; and makes a secondary determination on whether the target simulation system meets the standard based on the corrected overall anomalous rate. The attribution analysis module, which is connected to the data analysis module and the misjudgment correction module, determines whether the multimodal deep learning model in the target interview simulation system is abnormal based on the performance characteristic representation value corresponding to several abnormal comprehensive performance characteristic indices, and generates evaluation results. The facial expression feature parameters include: the similarity between the standard interviewee's image and text feature vectors and the stability of their emotional state; the text expression feature parameters include: the accuracy score of the response and the keyword matching degree of the resume; the interaction expression feature parameters include: the response time of the interview questions and the coherence of multiple rounds of question and answer.

2. The deep learning-based interview simulation analysis system according to claim 1, characterized in that, The data analysis module determines the facial expression characteristic representation value of each standard interviewee based on the first facial expression characteristic parameter and the second facial expression characteristic parameter. The text performance feature representation values ​​of each standard interviewee are determined based on the first text performance feature parameter and the second text performance feature parameter. The interaction performance characteristic representation value of each standard interviewee is determined based on the first interaction performance characteristic parameter and the second interaction performance characteristic parameter. The first facial expression feature parameter and the second facial expression feature parameter are respectively calculated by the similarity of the portrait-text feature vectors of each standard interviewee and the stability of the emotional state. The first text performance feature parameter and the second text performance feature parameter are calculated using the accuracy scores of each standard interviewee's responses and the keyword matching degree of the resume, respectively. The first interaction performance feature parameter and the second interaction performance feature parameter are calculated by the response time of each standard interviewee to the interview questions and the continuity of multiple rounds of question and answer.

3. The deep learning-based interview simulation analysis system according to claim 2, characterized in that, The data analysis module determines the comprehensive performance characteristic index of each standard interviewee by weighted summation of the first comprehensive performance characteristic parameter, the second comprehensive performance characteristic parameter, and the third comprehensive performance characteristic parameter. The first comprehensive performance feature parameter, the second comprehensive performance feature parameter, and the third comprehensive performance feature parameter are calculated using the facial expression feature value, text expression feature value, and interaction performance feature value of each standard interviewee, respectively.

4. The deep learning-based interview simulation analysis system according to claim 3, characterized in that, The standard evaluation module is used to determine that the comprehensive standard characteristic index is abnormal when the comprehensive performance characteristic index is less than a preset comprehensive performance characteristic index threshold.

5. The deep learning-based interview simulation analysis system according to claim 4, characterized in that, The standard evaluation module initially determines that the target interview simulation system does not meet the standard based on the fact that the overall abnormality rate of a number of standard interviewees is greater than a preset abnormality rate threshold.

6. The deep learning-based interview simulation analysis system according to claim 1, characterized in that, The misjudgment correction module acquires comprehensive performance feature information corresponding to several abnormal comprehensive performance feature indices, and extracts outlier data from the comprehensive performance feature information using the interquartile range method.

7. The deep learning-based interview simulation analysis system according to claim 6, characterized in that, The misjudgment correction module is used to remove the comprehensive performance characteristic index corresponding to the outlier data from a number of abnormal comprehensive performance characteristic indices, thereby correcting the overall abnormality rate of a number of standard interviewees.

8. The deep learning-based interview simulation analysis system according to claim 7, characterized in that, The misjudgment correction module determines that the target interview simulation system does not meet the standard based on the overall abnormality rate after correction being greater than the preset abnormality rate threshold.

9. The deep learning-based interview simulation analysis system according to claim 1, characterized in that, The attribution analysis module determines whether the multimodal deep learning model in the target interview simulation system is abnormal based on the mean of the performance characteristic representation values ​​corresponding to the comprehensive performance characteristic index of several anomalies after elimination.

10. The deep learning-based interview simulation analysis system according to claim 9, characterized in that, The attribution analysis module determines that there is an anomaly in the facial expression analysis in the multimodal deep learning model when the mean value of the facial expression feature representation is less than the preset threshold value of the facial expression feature representation. If the mean value of the text performance feature is less than the preset threshold value of the text performance feature, it is determined that there is an anomaly in the text understanding and evaluation in the multimodal deep learning model; if the mean value of the interaction performance feature is less than the preset threshold value of the interaction performance feature, it is determined that there is an anomaly in the interaction evaluation in the multimodal deep learning model.

Images