A simulated interview teaching method and system based on AI big model

Abstract

The present invention provides a simulated interview teaching method and system based on an AI big model, the method comprising: obtaining interview data and a face image of a user to be interviewed; obtaining an expression description text from a preset expression open text pool, and inputting the face image and the expression description text into a multimodal expression recognition submodel respectively, obtaining a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition submodel; performing similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result; inputting academic qualification data and resume data into the AI ​​big model, respectively obtaining an academic qualification rating value and a resume rating value output by the AI ​​big model; and determining a simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification rating value and the resume rating value. The present invention improves the accuracy of the simulated interview result.

Description

Technical Field

[0001] The present invention relates to the field of AI intelligent technology, and in particular to a simulation interview teaching method and system based on an AI large model. Background Art

[0002] Interview is an essential part of the workplace. Before applying for a job, the interviewee will sign up for a mock interview. The current mock interview method is mainly manual interview, that is, manual review of resumes, manual review of academic qualifications, and manual observation of the interviewee's expression. However, in the process of manual interview, there are often subjective assumptions, which leads to great differences in the final mock interview results, making the mock interview results inaccurate. Summary of the invention

[0003] In view of the technical problems existing in the prior art, the present invention provides a simulation interview teaching method and a system based on an AI big model, so as to improve the accuracy of the simulation interview results.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: a simulated interview teaching method based on an AI big model, the AI ​​big model includes a multimodal expression recognition sub-model, an academic qualification evaluation sub-model and a resume evaluation sub-model; the multimodal expression recognition sub-model is trained based on sample face images obtained under non-cooperative conditions and their corresponding sample expression description texts; the academic qualification evaluation sub-model is trained based on sample academic qualification data and their corresponding score values, and the resume evaluation sub-model is trained based on sample resume data and their corresponding score values, including:

[0005] Obtaining interview data and facial images of the user to be interviewed; the interview data includes academic qualifications and resume data;

[0006] Acquire an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model;

[0007] Performing a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result;

[0008] Input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score and resume score output by the AI ​​big model respectively;

[0009] Based on the expression recognition result, the academic qualification score and the resume score, a simulation interview teaching result of the user to be interviewed is determined.

[0010] According to a simulated interview teaching method based on an AI large model provided by the present invention, the multimodal expression recognition sub-model includes: a portrait feature extraction module and the expression semantic analysis module;

[0011] The step of inputting the facial image and the expression description text into a multimodal expression recognition sub-model respectively, and obtaining a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model, comprises:

[0012] Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module;

[0013] The expression description text is input into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain the text feature representation vector output by the expression semantic analysis module.

[0014] According to a simulated interview teaching method based on an AI large model provided by the present invention, the face image is input into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module, including:

[0015] Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model, performing mapping calculation on the entire facial image, and extracting a global feature vector of the facial image;

[0016] Performing mapping calculation on the target part of the face image to extract the local feature vector of each part in the face image;

[0017] The global feature vector and the local feature vector are connected to obtain a portrait feature representation vector of the face image.

[0018] According to a simulated interview teaching method based on an AI large model provided by the present invention, the expression description text is input into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain a text feature representation vector output by the expression semantic analysis module, including:

[0019] Inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition sub-model, analyzing the expression description sentences in the expression description text, and extracting expression description keywords;

[0020] Linear projection calculation and correlation calculation are performed on the expression description keywords to map the expression description text into a feature representation vector with the same dimension as the portrait feature representation vector, so as to obtain a text feature representation vector.

[0021] According to a simulated interview teaching method based on an AI large model provided by the present invention, determining the expression recognition result corresponding to the face image based on the similarity result includes:

[0022] Obtaining a maximum value of similarity from the similarity results, and when the maximum value of similarity is greater than or equal to a preset similarity threshold, using the expression description text corresponding to the maximum value of similarity as the expression recognition result corresponding to the face image; and / or,

[0023] Perform a maximum analysis on the similarity result to obtain a maximum value of the similarity; when the maximum value of the similarity is greater than or equal to a preset ratio of the maximum value of the similarity, determine that the expression recognition result corresponding to the face image is a composite expression, and use the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity as the expression recognition result corresponding to the face image.

[0024] According to a simulated interview teaching method based on an AI large model provided by the present invention, a similarity analysis is performed on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, including:

[0025] The portrait feature representation vector and the text feature representation vector are multiplied to obtain a target feature representation vector, the target feature representation vector is input into a preset multi-classification algorithm model to obtain a corresponding similarity score, and the similarity score is used as a similarity result.

[0026] According to a simulated interview teaching method based on an AI large model provided by the present invention, based on the expression recognition result, the academic qualification score and the resume score, the simulated interview teaching result of the user to be interviewed is determined, including:

[0027] Determine the expression score of the user to be interviewed based on the expression recognition result;

[0028] Determine the interview position of the user to be interviewed based on the resume data; the interview position includes customer service position, technical position, sales position and accounting position;

[0029] Based on the interview position, the expression score, the education score and the resume score, a simulation interview teaching result of the user to be interviewed is determined.

[0030] The present invention also provides a simulated interview teaching system based on an AI big model, wherein the AI ​​big model includes a multimodal expression recognition submodel, an academic qualification evaluation submodel and a resume evaluation submodel; the multimodal expression recognition submodel is trained based on sample face images obtained under non-cooperative conditions and their corresponding sample expression description texts; the academic qualification evaluation submodel is trained based on sample academic qualification data and their corresponding score values, and the resume evaluation submodel is trained based on sample resume data and their corresponding score values, and includes:

[0031] An acquisition module is used to acquire interview material data and face images of the user to be interviewed; the interview material data includes educational background data and resume data;

[0032] A first determination module is used to obtain an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model;

[0033] A second determination module is used to perform a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determine an expression recognition result corresponding to the face image based on the similarity result;

[0034] A third determination module is used to input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score value and resume score value output by the AI ​​big model respectively;

[0035] The simulated interview teaching module is used to determine the simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score value and the resume score value.

[0036] The present invention also provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing any of the above-mentioned simulated interview teaching methods based on the AI ​​large model.

[0037] The present invention also provides a non-transitory computer-readable storage medium, characterized in that a computer software program is stored in the storage medium, and when the computer software program is executed by a processor, it implements any of the above-mentioned simulated interview teaching methods based on the AI ​​big model.

[0038] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned simulated interview teaching methods based on the AI ​​big model.

[0039] The beneficial effects of the present invention are as follows: through the multimodal expression recognition sub-model, academic qualification evaluation sub-model and resume evaluation sub-model in the AI ​​large model, the expression recognition result, academic qualification score value and resume score value of the user to be interviewed are automatically obtained according to the interview material data and facial image of the user to be interviewed; finally, the simulated interview teaching result of the user to be interviewed is determined according to the expression recognition result, academic qualification score value and resume score value; the whole process is evaluated by a pre-trained high-precision model, does not require manual review, avoids human interference factors, and improves the accuracy of the simulated interview results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is one of the flow charts of the simulated interview teaching method based on the AI ​​big model provided by the present invention;

[0041] Figure 2 This is the second flow chart of the simulated interview teaching method based on the AI ​​big model provided by the present invention;

[0042] Figure 3 It is a structural diagram of the simulated interview teaching system based on the AI ​​big model provided by the present invention;

[0043] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present invention;

[0044] Figure 5 A schematic diagram of an embodiment of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0046] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0047] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0048] Combine the following Figure 1 The present invention describes a simulated interview teaching method based on an AI large model, which specifically includes steps 10 to 50:

[0049] It should be noted that the AI ​​big model in the embodiment of the present invention includes a multimodal expression recognition sub-model, an academic qualification assessment sub-model and a resume assessment sub-model; the multimodal expression recognition sub-model is trained based on sample facial images obtained under non-cooperative conditions and their corresponding sample expression description texts; the academic qualification assessment sub-model is trained based on sample academic qualification material data and its corresponding score values, and the resume assessment sub-model is trained based on sample resume material data and its corresponding score values.

[0050] In one embodiment, {the academic record data is a doctorate, the academic record score is A1}, {the academic record data is a master's degree, the academic record score is A2}, {the academic record data is a bachelor's degree, the academic record score is A3}, {the academic record data is a junior college, the academic record score is A4}, {the academic record data is other, the academic record score is A5} are trained to obtain the academic record evaluation sub-model. {the resume typesetting result is excellent, the resume score is B1}, {the resume typesetting result is good, the resume score is B2}, {the resume typesetting result is qualified, the resume score is B3}, {the resume typesetting result is unqualified, the resume score is B4} are trained to obtain the resume evaluation sub-model.

[0051] Step 10, obtaining the interview data and face image of the user to be interviewed; the interview data includes academic qualifications data and resume data.

[0052] Optionally, when conducting a mock interview teaching, the user to be interviewed needs to input interview material data into the mock interview teaching system, so that the interview material data of the user to be interviewed can be obtained, and the interview material data includes academic information data and resume information data. At the same time, the mock interview teaching system collects the face image of the user to be interviewed through a camera device.

[0053] Step 20, obtaining expression description text from a preset expression open text pool, and inputting the face image and the expression description text into the multimodal expression recognition sub-model respectively, to obtain the portrait feature representation vector and the text feature representation vector respectively output by the multimodal expression recognition sub-model.

[0054] Optionally, before executing this step, you need to pre-set Figure 2 The expression open text pool shown is used to store expression description texts, and the expression description texts may include expression description keywords or expression description sentences.

[0055] In an embodiment of the present invention, the multimodal expression recognition submodel includes: a portrait feature extraction module and an expression semantic analysis module. The face image and the expression description text are respectively input into the multimodal expression recognition submodel to obtain the portrait feature representation vector and the text feature representation vector respectively output by the multimodal expression recognition submodel. The corresponding implementation process includes: inputting the face image into the portrait feature extraction module of the multimodal expression recognition submodel to obtain the portrait feature representation vector output by the portrait feature extraction module; inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition submodel to obtain the text feature representation vector output by the expression semantic analysis module.

[0056] Among them, the face image is input into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module. The corresponding specific implementation process includes: inputting the face image into the portrait feature extraction module of the multimodal expression recognition sub-model, performing mapping calculation on the whole face image, and extracting the global feature vector of the face image; and performing mapping calculation on the target part of the face image, and extracting the local feature vector of each part in the face image; connecting the global feature vector and the local feature vector to obtain the portrait feature representation vector of the face image.

[0057] The expression description text is input into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain the text feature representation vector output by the expression semantic analysis module. The corresponding specific implementation process includes: inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition sub-model, analyzing the expression description sentences in the expression description text, extracting the expression description keywords, performing linear projection calculation and correlation calculation on the expression description keywords to map the expression description text into a feature representation vector of the same dimension as the portrait feature representation vector to obtain the text feature representation vector.

[0058] The training and recognition process of the multimodal expression recognition sub-model includes the following steps: (1) selecting a preset portrait expression pairing sample pool (i.e. Figure 2The sample face images with different expressions are randomly selected in batches from the portrait-expression pairing sample pool in Figure 2 The facial images in the image) and their corresponding sample expression description texts (i.e. Figure 2 (2) using the portrait feature extraction module to perform convolution, linear mapping, nonlinear activation and other mapping calculations on the selected sample face images; (3) using the expression semantic analysis module (i.e. Figure 2 (4) Utilize the preset portrait expression semantic association module to calculate the similarity between the portrait sample mapping features (i.e., portrait feature representation vector) and the expression semantic features (i.e., text feature representation vector); (5) Utilize the expression reasoning module to match the sample expression description text (i.e., expression semantic expression) corresponding to the sample portrait (i.e., sample face image) according to the similarity, and the sample expression description text includes expression description keywords / sentences. When the sample face image and the sample expression description text are consistent, the similarity should be as high as possible; when the sample face image and the sample expression description text are inconsistent, the similarity should be as low as possible, and the corresponding loss function is constructed according to this criterion; (6) Repeat steps (1)-(5) until the similarity loss function converges or reaches the maximum number of iterations, and obtain the mapping model parameters of the portrait feature extraction module and the expression semantic analysis module (i.e., the model parameters of the multimodal expression recognition sub-model); (7) From the expression description text in the preset expression open text pool, use the expression description text of the trained multimodal expression recognition sub-model to obtain the expression description text of the sample face image. The emotion semantic analysis module generates a set of expression text representation vectors (i.e., a set of text feature representation vectors); (8) selects face images to be analyzed in an open environment in batches or individually from the preset sample pool to be analyzed; (9) uses the portrait feature extraction module of the trained multimodal expression recognition sub-model to extract the feature vector of the portrait modality (i.e., the portrait feature representation vector) from the selected face image, and calculates the similarity with the expression text representation vector (i.e., the text feature representation vector) obtained in step (7) in turn, and uses the expression reasoning module to obtain the expression recognition result, which is stored in the analyzed sample pool. In simple cases, the expression description text corresponding to the vector with the highest similarity can be directly selected as the expression recognition result. (10) Repeat steps (8)-(9) until there are no remaining face images in the sample pool to be analyzed.

[0059] For example, the training process of the multimodal expression recognition sub-model is as follows:

[0060] Step 1: Randomly select portrait-text pairs B from the portrait-expression pairing sample pool φ k , t k , l k Step 2: Input fk into the portrait feature extraction module to obtain the portrait feature representation vector v kSpecifically, fk is first normalized to an image of the same size as F k , for F k Perform mapping transformation to obtain the global feature vector of the global face image The face image is input into the portrait feature extraction module of the multimodal expression recognition sub-model, and the whole face image is mapped and calculated to extract the global feature vector of the face image; fk is divided into different regions and the sub-images of each region are normalized into sub-images of the same size, and the local feature vector of the local face image is obtained by mapping transformation calculation. That is, mapping calculation is performed on the target part of the face image to extract the local feature vector of each part of the face image. and the local eigenvector Connect them together and calculate through linear projection to obtain the portrait feature representation vector v of the d-dimensional portrait k Step 3: Set t k Input to the expression text mapping module to obtain the text feature representation vector w k Specifically, for the expression description keywords in the expression description text, a preset template is used to organize them into descriptive sentences (i.e., expression description sentences). k The dimension and v k Keep it consistent, all are d-dimensional. Step 4: Calculate the similarity s of the feature representation vectors of different modalities within the batch portrait-text pairs i，j , where i and j represent the index of the face image and the index of the expression description text respectively, and the similarity s i，j The vector v is represented by the portrait feature i and the expression text feature representation vector (i.e., text feature representation vector) w j Calculate and obtain. Step 5: Use the similarity matrix S in the batch = {s i，j}Calculate the loss function. Optimize the parameters of the portrait feature extraction module and the expression text mapping module with the minimization of the loss function as the optimization goal. When the face image and the expression description text come from the same sample pair, the similarity is close to 1; when the face image and the expression description text come from different sample pairs, the similarity is close to 0. Step 6: Iterate steps 1-5 until the loss function no longer decreases or the maximum number of iterations is reached.

[0061] It should be noted that the method for constructing the portrait expression pairing sample pool is as follows: Step 1: Collect scene images containing faces under non-cooperative conditions. Step 2: Use a preset face detector to detect faces in the scene image. Step 3: Cut the detected faces out of the scene image to obtain a face image fh. Step 4: For the face image, annotate the expression description sentences t related to the facial expression h , establish a standard portrait expression description dataset φh ={f h , t h Step 5: Considering the high cost of manual annotation, we can crawl expression description keywords related to expressions from the Internet. w and its corresponding face image f w , establish the portrait expression association dataset φ w ={f w , t w}. Step 6: Set the standard portrait expression description dataset φ h and φ w The samples are merged into a portrait expression pairing sample pool φ = {f, t, l}, where 1 represents the flag bit of the data source.

[0062] Step 30, performing similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result.

[0063] A similarity analysis is performed on the portrait feature representation vector and the text feature representation vector to obtain a similarity result. Specifically, the portrait feature representation vector and the text feature representation vector may be multiplied to obtain a target feature representation vector, and the target feature representation vector is input into a preset multi-classification algorithm model to obtain a corresponding similarity score, and the similarity score is used as the similarity result.

[0064] Specifically, the maximum value of the similarity can be obtained from the similarity result. When the maximum value of the similarity is greater than or equal to a preset similarity threshold, the expression description text corresponding to the maximum value of the similarity is used as the expression recognition result corresponding to the face image. The similarity result is subjected to maximum value analysis to obtain the maximum value of the similarity; when the maximum value of the similarity is greater than or equal to a preset ratio of the maximum value of the similarity, the expression recognition result corresponding to the face image is determined to be a composite expression, and the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity are both used as the expression recognition result corresponding to the face image.

[0065] In an embodiment of the present invention, the expression analysis and recognition method for the expression text open pool is as follows: Step 1: Set the expression text open pool according to the actual application scenario. Step 2: Use the expression semantic analysis module to generate an expression text representation matrix M from the expression description text (including expression description keywords or expression description sentences) in the expression open text pool, where M has d rows and n columns, d is the output dimension of the expression text mapping module, nis the number of expression description texts contained in the expression open text pool. That is, each row of M represents the mapping vector (i.e., text feature representation vector) of each expression description text in the expression open text pool. Step 3: Select the face image fk of the expression to be recognized from the sample pool to be analyzed. Step 4: Input the face image fk into the portrait feature extraction module to obtain the portrait feature representation vector v k , represented by a d-dimensional column vector. Wherein, step 20 corresponds to the above steps 1 to 4. In addition, step 30 corresponds to the above step 5. Wherein, step 5: the expression text representation matrix M and the portrait feature representation vector v k Multiply them together to get an n-dimensional vector. Then transform it through Softmax to get the similarity score s k In addition, step 30 corresponds to the above steps 6-7. Step 6: According to the similarity score s k The expression recognition results are obtained and stored in the analyzed sample pool. Specifically, if s k If the maximum value of s is greater than or equal to the preset similarity, the expression recognition result is recorded as the expression description text corresponding to the maximum value; if s k If the maximum value is greater than or equal to a preset proportion of the maximum value (e.g. 80%), the expression recognition result is recorded as a composite expression, and the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity are both used as the expression recognition results corresponding to the face image. And so on. Step 7: Iterate steps 3-6 until there are no face images with expressions to be recognized in the sample pool to be analyzed.

[0066] Take the analysis of emotional changes of a group of students waiting for interviews on a certain campus as an example:

[0067] (1) Set up an open text pool of expressions. When focusing on the negative emotions of the group of students to be interviewed, set the open text pool of expressions to "normal expressions" and "negative expressions". "Negative expressions" can be further refined into "unhappy, angry, sad, numb", etc. (2) Use the expression text mapping module to map the relevant expression description text in the open text pool of expressions to expression text modal features (i.e., text feature representation vectors) one-to-one. (3) Obtain scene images through cameras on campus, and further obtain facial images of the group, and store them in the sample pool to be analyzed. (4) Select the sample to be analyzed from the sample pool to be analyzed (i.e., the facial image in the open environment to be analyzed), and use the portrait feature extraction module to extract the portrait modal features (i.e., the portrait feature representation vector). (5) Use the portrait expression semantic association model (i.e., the portrait expression semantic association model) to calculate the similarity between the portrait modal features and the expression text modal features. (6) Perform maximum value analysis on the similarity to obtain the expression analysis results, and pair them with the corresponding facial images of the students to be interviewed and save them in the analyzed sample pool. (7) Through subsequent face recognition and other modules, the expression changes and emotion analysis results of the same interviewed student at different times are summarized and sorted to achieve dynamic emotion analysis.

[0068] Step 40, input the academic qualification data and resume data into the AI ​​big model, and obtain the academic qualification score and resume score output by the AI ​​big model respectively.

[0069] Optionally, the academic qualification data is input into the academic qualification evaluation sub-model to obtain the academic qualification score value output by the academic qualification evaluation sub-model. In one embodiment, the academic qualification data is input into the academic qualification evaluation sub-model, and the academic qualification evaluation sub-model identifies the academic qualification data. If the academic qualification data is a doctorate, the output academic qualification score value is A1; if the academic qualification data is a master's degree, the output academic qualification score value is A2; if the academic qualification data is a bachelor's degree, the output academic qualification score value is A3; if the academic qualification data is a junior college degree, the output academic qualification score value is A4; if the academic qualification data is other, the output academic qualification score value is A5.

[0070] Optionally, the resume data is input into the resume evaluation sub-model to obtain the resume evaluation sub-model. In one embodiment, the resume data is input into the resume evaluation sub-model, and the resume evaluation sub-model identifies the layout of the resume data to obtain the resume layout result. If the resume layout result is excellent, the output resume score value is B1, if the resume layout result is good, the output resume score value is B2, if the resume layout result is qualified, the output resume score value is B3, and if the resume layout result is unqualified, the output resume score value is B4.

[0071] Step 50, based on the expression recognition result, the academic qualification score and the resume score, determine the simulated interview teaching result of the user to be interviewed.

[0072] Optionally, the results of the simulated interview teaching for the user to be interviewed are determined based on the expression recognition results, academic qualification rating values ​​and resume rating values, specifically: determining the expression rating value of the user to be interviewed based on the expression recognition results; determining the interview position of the user to be interviewed based on the resume data; interview positions include customer service positions, technical positions, sales positions and accounting positions; determining the results of the simulated interview teaching for the user to be interviewed based on the interview position, expression rating value, academic qualification rating value and resume rating value.

[0073] Optionally, the expression score of the user to be interviewed is determined based on the expression recognition result. In one embodiment, if the expression recognition result is a normal expression, the expression score is C1, and if the expression recognition result is a negative expression, the expression score is C2. Further, the interview position of the user to be interviewed is determined based on the resume data. The interview positions include customer service positions, technical positions, sales positions, and accounting positions.

[0074] Optionally, if the interview position is a customer service position, the mock interview teaching result of the user to be interviewed is: 70%*C+20%*B+10%*A(points). If the interview position is a technical position, the mock interview teaching result of the user to be interviewed is: 85%*B+10%*A+5%*C. If the interview position is a sales position, the mock interview teaching result of the user to be interviewed is: 90%*C+5%*B+5%*A. If the interview position is an accounting position, the mock interview teaching result of the user to be interviewed is: 80%*C+10%*B+10%*A, where A is the academic score, B is the resume score, and C is the relationship score.

[0075] The embodiment of the present invention uses the multimodal expression recognition sub-model, academic qualification evaluation sub-model and resume evaluation sub-model in the AI ​​large model to automatically obtain the expression recognition result, academic qualification score value and resume score value of the user to be interviewed based on the interview material data and facial image of the user to be interviewed, and finally determines the simulated interview teaching result of the user to be interviewed based on the expression recognition result, academic qualification score value and resume score value. The whole process is evaluated by a pre-trained high-precision model, does not require manual review, avoids human interference factors, and improves the accuracy of the simulated interview results.

[0076] The following is a description of the simulated interview teaching system based on the AI ​​big model provided by the present invention. The simulated interview teaching system based on the AI ​​big model described below and the simulated interview teaching method based on the AI ​​big model described above can be referred to each other. Figure 3 As shown, Figure 3 : is a structural diagram of a simulated interview teaching system based on an AI big model provided by the present invention. The simulated interview teaching system based on an AI big model includes:

[0077] Acquisition module 301, used to acquire interview material data and face image of the user to be interviewed; the interview material data includes academic qualification data and resume data;

[0078] The first determination module 302 is used to obtain an expression description text from a preset expression open text pool, input the face image and the expression description text into a multimodal expression recognition sub-model respectively, and obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model;

[0079] The second determination module 303 is used to perform similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determine the expression recognition result corresponding to the face image based on the similarity result;

[0080] The third determination module 304 is used to input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score value and resume score value output by the AI ​​big model respectively;

[0081] The simulated interview teaching module 305 is used to determine the simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score value and the resume score value.

[0082] This embodiment uses the multimodal expression recognition sub-model, academic qualification assessment sub-model and resume assessment sub-model in the AI ​​large model to automatically obtain the expression recognition results, academic qualification rating values ​​and resume rating values ​​of the user to be interviewed based on the interview data data and facial images of the user to be interviewed. Finally, the simulated interview teaching results of the user to be interviewed are determined based on the expression recognition results, academic qualification rating values ​​and resume rating values. The entire process is evaluated through a pre-trained high-precision model, does not require manual review, avoids human interference factors, and improves the accuracy of the simulated interview results.

[0083] In an optional embodiment, the first determining module 302 is further configured to:

[0084] Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module;

[0085] The expression description text is input into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain the text feature representation vector output by the expression semantic analysis module.

[0086] In an optional embodiment, the first determining module 302 is further configured to:

[0087] Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model, performing mapping calculation on the entire facial image, and extracting a global feature vector of the facial image;

[0088] Performing mapping calculation on the target part of the face image to extract the local feature vector of each part in the face image;

[0089] The global feature vector and the local feature vector are connected to obtain a portrait feature representation vector of the face image.

[0090] In an optional embodiment, the first determining module 302 is further configured to:

[0091] Inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition sub-model, analyzing the expression description sentences in the expression description text, and extracting expression description keywords;

[0092] Linear projection calculation and correlation calculation are performed on the expression description keywords to map the expression description text into a feature representation vector with the same dimension as the portrait feature representation vector, so as to obtain a text feature representation vector.

[0093] In an optional embodiment, the second determining module 303 is further configured to:

[0094] Obtaining a maximum value of similarity from the similarity results, and when the maximum value of similarity is greater than or equal to a preset similarity threshold, using the expression description text corresponding to the maximum value of similarity as the expression recognition result corresponding to the face image; and / or,

[0095] Perform a maximum analysis on the similarity result to obtain a maximum value of the similarity; when the maximum value of the similarity is greater than or equal to a preset ratio of the maximum value of the similarity, determine that the expression recognition result corresponding to the face image is a composite expression, and use the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity as the expression recognition result corresponding to the face image.

[0096] In an optional embodiment, the second determining module 303 is further configured to:

[0097] The portrait feature representation vector and the text feature representation vector are multiplied to obtain a target feature representation vector, the target feature representation vector is input into a preset multi-classification algorithm model to obtain a corresponding similarity score, and the similarity score is used as a similarity result.

[0098] In an optional embodiment, the mock interview teaching module 305 is also used to:

[0099] Determine the expression score of the user to be interviewed based on the expression recognition result;

[0100] Determine the interview position of the user to be interviewed based on the resume data; the interview position includes customer service position, technical position, sales position and accounting position;

[0101] Based on the interview position, the expression score, the education score and the resume score, a simulation interview teaching result of the user to be interviewed is determined.

[0102] See also Figure 4 , Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device 400, including a memory 410, a processor 420, and a computer program 411 stored in the memory 410 and executable on the processor 420. When the processor 420 executes the computer program 411, the following steps are implemented:

[0103] Obtaining interview data and facial images of the user to be interviewed; the interview data includes academic qualifications and resume data;

[0104] Acquire an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model;

[0105] Performing a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result;

[0106] Input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score and resume score output by the AI ​​big model respectively;

[0107] Based on the expression recognition result, the academic qualification score and the resume score, a simulation interview teaching result of the user to be interviewed is determined.

[0108] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 500, on which a computer program 511 is stored. When the computer program 511 is executed by a processor, the following steps are implemented:

[0109] Obtaining interview data and facial images of the user to be interviewed; the interview data includes academic qualifications and resume data;

[0110] Acquire an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model;

[0111] Performing a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result;

[0112] Input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score and resume score output by the AI ​​big model respectively;

[0113] Based on the expression recognition result, the academic qualification score and the resume score, a simulation interview teaching result of the user to be interviewed is determined.

[0114] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0115] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0120] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A simulated interview teaching method based on AI big model, characterized in that: The AI ​​big model includes a multimodal expression recognition sub-model, an academic qualification assessment sub-model and a resume assessment sub-model; the multimodal expression recognition sub-model is trained based on sample face images obtained under non-cooperative conditions and their corresponding sample expression description texts; The academic qualification evaluation sub-model is trained based on the sample academic qualification data and its corresponding score values, and the resume evaluation sub-model is trained based on the sample resume data and its corresponding score values, including Obtaining interview data and facial images of the user to be interviewed; the interview data includes academic qualifications data and resume data; Acquire an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model; Performing a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determining an expression recognition result corresponding to the face image based on the similarity result; Input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score and resume score output by the AI ​​big model respectively; Determine the simulation interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score and the resume score; The method for constructing a sample pool of portrait expression pairings is as follows: Step 1: Collect scene images containing faces under non-cooperative conditions; Step 2: Use a preset face detector to detect faces in the scene image; Step 3: Cut the detected faces out of the scene image to obtain a face image f h ; Step 4: For the face image, mark the expression description sentence related to the facial expression t h , establish a standard portrait expression description dataset φ h ={f h ,t h }; Step 5: Crawl emoticon-related emoticon description keywords from the Internet w and its corresponding face image f w , establish the portrait expression association dataset φ w ={f w ,t w }; Step 6: Set the standard portrait expression description dataset φ h And the portrait expression association dataset φ w The samples are merged into a portrait expression pairing sample pool φ = {f, t, l}, where l represents the flag bit of the data source; The training process of the multimodal expression recognition sub-model is as follows: Step 1: Randomly select portrait-text pairs B from the portrait-expression pairing sample pool φ k ,t k ,l k }; Step 2: Transform the face image f k Input to the portrait feature extraction module to obtain the portrait feature representation vector v k Specifically, f k First normalize to the same size image F k , for F k Perform mapping transformation to obtain the global feature vector of the global face image That is, the face image is input into the portrait feature extraction module of the multimodal expression recognition sub-model, the whole face image is mapped and calculated, and the global feature vector of the face image is extracted; f k Divide into different regions and normalize the sub-images of each region into sub-images of the same size, and perform mapping transformation to obtain the local feature vector of the local face image That is, mapping calculation is performed on the target part of the face image to extract the local feature vectors of each part of the face image; the global feature vector and the local eigenvector Connect them together and calculate through linear projection to obtain the portrait feature representation vector v of the d-dimensional portrait k ; Step 3: Translate the expression description text k Input to the expression text mapping module to obtain the text feature representation vector w k Specifically, for the expression description keywords in the expression description text, the preset template is used to organize them into expression description sentences. k The dimension and v k Keep it consistent, all are d-dimensional; Step 4: Calculate the similarity s of the feature representation vectors of different modes within the batch portrait-text pairs i,j , where i and j represent the index of the face image and the index of the expression description text respectively, and the similarity s i,j The vector v is represented by the portrait feature i and the expression text feature representation vector w j Calculation acquisition; Step 5: Using the similarity matrix S in the batch = {s i,j }Calculate the loss function, and optimize the parameters of the portrait feature extraction module and the expression text mapping module with the minimization of the loss function as the optimization goal; when the face image and the expression description text come from the same sample pair, the similarity is close to 1; when the face image and the expression description text come from different sample pairs, the similarity is close to 0; Step 6: Iterate steps 1-5 until the loss function no longer decreases or the maximum number of iterations is reached; The expression analysis and recognition method for the expression text open pool is as follows: Step 1: Set the expression text open pool according to the actual application scenario; Step 2: Use the expression semantic analysis module to generate an expression text representation matrix M from the expression description text in the expression open text pool, where M has d rows and n columns, d is the output dimension of the expression text mapping module, and n is the number of expression description texts contained in the expression open text pool; Step 3: Select a face image f of the expression to be recognized from the sample pool to be analyzed k ; Step 4: Place the portrait f k Input to the portrait feature extraction module to obtain the portrait feature representation vector v k , represented by a d-dimensional column vector; Step 5: Combine the expression text representation matrix M with the portrait feature representation vector v k Multiply them together to get an n-dimensional vector; transform it through Softmax to get the similarity score s k ; Step 6: Based on the similarity score s k Get the expression recognition results and store them in the analyzed sample pool. Specifically: if s k If the maximum value of s is greater than or equal to the preset similarity, the expression recognition result is recorded as the expression description text corresponding to the maximum value; if s k If the maximum value of is greater than or equal to the preset ratio of the maximum value, the expression recognition result is recorded as a composite expression, and the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity are both used as the expression recognition result corresponding to the face image; Step 7: Iterate steps 3-6 until there is no face image with an expression to be recognized in the sample pool to be analyzed; The step of determining the simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score and the resume score includes: Determine the expression score of the user to be interviewed based on the expression recognition result; Determine the interview position of the user to be interviewed based on the resume data; the interview position includes customer service position, technical position, sales position and accounting position; If the interview position is a customer service position, the mock interview teaching result of the user to be interviewed is: 70%*C+20%*B+10%*A; if the interview position is a technical position, the mock interview teaching result of the user to be interviewed is: 85%*B+10%*A+5%*C; if the interview position is a sales position, the mock interview teaching result of the user to be interviewed is: 90%*C+5%*B+5%*A; if the interview position is an accounting position, the mock interview teaching result of the user to be interviewed is: 80%*C+10%*B+10%*A, where A is the academic qualification score, B is the resume score, and C is the relationship score.

2. The AI ​​big model-based simulated interview teaching method according to claim 1 is characterized in that: The multimodal expression recognition sub-model includes: a portrait feature extraction module and the expression semantic analysis module; The step of inputting the facial image and the expression description text into a multimodal expression recognition sub-model respectively, and obtaining a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model, comprises: Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module; The expression description text is input into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain the text feature representation vector output by the expression semantic analysis module.

3. The AI ​​big model-based simulated interview teaching method according to claim 2 is characterized in that: The step of inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model to obtain the portrait feature representation vector output by the portrait feature extraction module comprises: Inputting the facial image into the portrait feature extraction module of the multimodal expression recognition sub-model, performing mapping calculation on the entire facial image, and extracting a global feature vector of the facial image; Performing mapping calculation on the target part of the face image to extract the local feature vector of each part in the face image; The global feature vector and the local feature vector are connected to obtain a portrait feature representation vector of the face image.

4. The AI ​​big model-based simulated interview teaching method according to claim 2 is characterized in that: The step of inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition sub-model to obtain the text feature representation vector output by the expression semantic analysis module comprises: Inputting the expression description text into the expression semantic analysis module of the multimodal expression recognition sub-model, analyzing the expression description sentences in the expression description text, and extracting expression description keywords; Linear projection calculation and correlation calculation are performed on the expression description keywords to map the expression description text into a feature representation vector with the same dimension as the portrait feature representation vector, so as to obtain a text feature representation vector.

5. The AI ​​big model-based simulated interview teaching method according to claim 1 is characterized in that: The determining the expression recognition result corresponding to the face image based on the similarity result includes: Obtaining a maximum value of similarity from the similarity results, and when the maximum value of similarity is greater than or equal to a preset similarity threshold, using the expression description text corresponding to the maximum value of similarity as the expression recognition result corresponding to the face image; and / or, Perform a maximum analysis on the similarity result to obtain a maximum value of the similarity; when the maximum value of the similarity is greater than or equal to a preset ratio of the maximum value of the similarity, determine that the expression recognition result corresponding to the face image is a composite expression, and use the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity as the expression recognition result corresponding to the face image.

6. The AI ​​big model-based simulated interview teaching method according to claim 1 is characterized in that: The performing similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result includes: The portrait feature representation vector and the text feature representation vector are multiplied to obtain a target feature representation vector, the target feature representation vector is input into a preset multi-classification algorithm model to obtain a corresponding similarity score, and the similarity score is used as a similarity result.

7. A simulated interview teaching system based on AI big model, characterized in that: The AI ​​big model includes a multimodal expression recognition sub-model, an academic qualification assessment sub-model and a resume assessment sub-model; the multimodal expression recognition sub-model is trained based on sample face images obtained under non-cooperative conditions and their corresponding sample expression description texts; The educational qualification evaluation sub-model is obtained by training based on sample educational qualification data and its corresponding score values, and the resume evaluation sub-model is obtained by training based on sample resume data and its corresponding score values, including: An acquisition module is used to acquire interview material data and face images of the user to be interviewed; the interview material data includes educational background data and resume data; A first determination module is used to obtain an expression description text from a preset expression open text pool, and input the face image and the expression description text into a multimodal expression recognition sub-model respectively, to obtain a portrait feature representation vector and a text feature representation vector respectively output by the multimodal expression recognition sub-model; A second determination module is used to perform a similarity analysis on the portrait feature representation vector and the text feature representation vector to obtain a similarity result, and determine an expression recognition result corresponding to the face image based on the similarity result; A third determination module is used to input the academic qualification data and the resume data into the AI ​​big model, and obtain the academic qualification score value and resume score value output by the AI ​​big model respectively; A simulated interview teaching module, used to determine the simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score and the resume score; The method for constructing a sample pool of portrait expression pairings is as follows: Step 1: Collect scene images containing faces under non-cooperative conditions; Step 2: Use a preset face detector to detect faces in the scene image; Step 3: Cut the detected faces out of the scene image to obtain a face image f h ; Step 4: For the face image, mark the expression description sentence related to the facial expression t h , establish a standard portrait expression description dataset φ h ={f h ,t h }; Step 5: Crawl emoticon-related emoticon description keywords from the Internet w and its corresponding face image f w , establish the portrait expression association dataset φ w ={f w ,t w }; Step 6: Set the standard portrait expression description dataset φ h And the portrait expression association dataset φ w The samples are merged into a portrait expression pairing sample pool φ = {f, t, l}, where l represents the flag bit of the data source; The training process of the multimodal expression recognition sub-model is as follows: Step 1: Randomly select portrait-text pairs B from the portrait-expression pairing sample pool φ k ,t k ,l k }; Step 2: Transform the face image f k Input to the portrait feature extraction module to obtain the portrait feature representation vector v k Specifically, f k First normalize to the same size image F k , for F k Perform mapping transformation to obtain the global feature vector of the global face image That is, the face image is input into the portrait feature extraction module of the multimodal expression recognition sub-model, the whole face image is mapped and calculated, and the global feature vector of the face image is extracted; f k Divide into different regions and normalize the sub-images of each region into sub-images of the same size, and perform mapping transformation to obtain the local feature vector of the local face image That is, mapping calculation is performed on the target part of the face image to extract the local feature vectors of each part of the face image; the global feature vector and the local eigenvector Connect them together and calculate through linear projection to obtain the portrait feature representation vector v of the d-dimensional portrait k ; Step 3: Translate the expression description text k Input to the expression text mapping module to obtain the text feature representation vector w k Specifically, for the expression description keywords in the expression description text, the preset template is used to organize them into expression description sentences. k The dimension and v k Keep it consistent, all are d-dimensional; Step 4: Calculate the similarity s of the feature representation vectors of different modes within the batch portrait-text pairs i,j , where i and j represent the index of the face image and the index of the expression description text respectively, and the similarity s i,j The vector v is represented by the portrait feature i and the expression text feature representation vector w j Calculation acquisition; Step 5: Using the similarity matrix S in the batch = {s i,j }Calculate the loss function, and optimize the parameters of the portrait feature extraction module and the expression text mapping module with the minimization of the loss function as the optimization goal; when the face image and the expression description text come from the same sample pair, the similarity is close to 1; when the face image and the expression description text come from different sample pairs, the similarity is close to 0; Step 6: Iterate steps 1-5 until the loss function no longer decreases or the maximum number of iterations is reached; The expression analysis and recognition method for the expression text open pool is as follows: Step 1: Set the expression text open pool according to the actual application scenario; Step 2: Use the expression semantic analysis module to generate an expression text representation matrix M from the expression description text in the expression open text pool, where M has d rows and n columns, d is the output dimension of the expression text mapping module, and n is the number of expression description texts contained in the expression open text pool; Step 3: Select a face image f of the expression to be recognized from the sample pool to be analyzed k ; Step 4: Place the portrait f k Input to the portrait feature extraction module to obtain the portrait feature representation vector v k , represented by a d-dimensional column vector; Step 5: Combine the expression text representation matrix M with the portrait feature representation vector v k Multiply them together to get an n-dimensional vector; transform it through Softmax to get the similarity score s k ; Step 6: Based on the similarity score s k Get the expression recognition results and store them in the analyzed sample pool. Specifically: if s k If the maximum value of s is greater than or equal to the preset similarity, the expression recognition result is recorded as the expression description text corresponding to the maximum value; if s k If the maximum value of is greater than or equal to the preset ratio of the maximum value, the expression recognition result is recorded as a composite expression, and the first expression description text corresponding to the maximum value of the similarity and the second expression description text corresponding to the maximum value of the similarity are both used as the expression recognition result corresponding to the face image; Step 7: Iterate steps 3-6 until there is no face image with an expression to be recognized in the sample pool to be analyzed; The step of determining the simulated interview teaching result of the user to be interviewed based on the expression recognition result, the academic qualification score and the resume score includes: Determine the expression score of the user to be interviewed based on the expression recognition result; Determine the interview position of the user to be interviewed based on the resume data; the interview position includes customer service position, technical position, sales position and accounting position; If the interview position is a customer service position, the mock interview teaching result of the user to be interviewed is: 70%*C+20%*B+10%*A; if the interview position is a technical position, the mock interview teaching result of the user to be interviewed is: 85%*B+10%*A+5%*C; if the interview position is a sales position, the mock interview teaching result of the user to be interviewed is: 90%*C+5%*B+5%*A; if the interview position is an accounting position, the mock interview teaching result of the user to be interviewed is: 80%*C+10%*B+10%*A, where A is the academic qualification score, B is the resume score, and C is the relationship score.

8. An electronic device, characterized in that: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the simulated interview teaching method based on the AI ​​big model as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, which, when executed by a processor, implements the simulated interview teaching method based on the AI ​​big model as described in any one of claims 1 to 6.

Images