Interview processing method and device based on digital human, storage medium and program product

Abstract

The embodiment of the invention provides an interview processing method and device based on a digital human, a storage medium and a program product. In the embodiment of the invention, in the online interview process, the listening material is introduced, so that the picture of the target interview room can be prevented from being still in the response process of the target job seeker. Furthermore, according to the first digital human interview material and the digital human listening material, a first transition frame material is generated, and after the first digital human interview material is played, the first transition frame material is switched to, and then the first transition frame material is switched to the digital human listening material. Smooth transition of digital human actions in the first digital human interview material and the digital human listening material can be realized. Therefore, not only can the smoothness of the online interview picture be improved, but also the richness and authenticity of the online interview interaction process can be improved, and the interview experience of job seekers is improved.

Description

Technical Field

[0001] This application relates to the field of Internet live broadcast technology, and in particular to an interview processing method, device, storage medium and program product based on digital humans. Background Art

[0002] In the job hunting process, in addition to submitting a resume, the interview is a crucial link. During the interview, the interviewer can judge the actual comprehensive ability of the job seeker based on the appearance, words and deeds, question answers and state of the job seeker. With the continuous development of network technology, the method of real-time online interview through digital humans has been widely used.

[0003] However, in the current digital human interview, the questioning process is fixed, without pertinence, and the fluency between different digital human pictures is poor, lacking interview authenticity, and the interview experience of job seekers is poor. Summary of the Invention

[0004] Multiple aspects of this application provide an interview video processing method, device, storage medium and program product based on digital humans to improve the richness and authenticity of the online interview interaction process and enhance the interview experience of job seekers.

[0005] An embodiment of this application provides an interview processing method based on digital humans, which is applied to a digital human server. The target job seeker conducts an online interview based on digital humans for a target position through a target interview room provided by the digital human server. The method includes:

[0006] During the online interview, according to the attribute information of the target job seeker or the response information of the previous interview question, determine the first digital human interview material, and the first digital human interview material includes the first interview question; send the first digital human interview material to the job seeker interview terminal for the job seeker interview terminal to play the first digital human interview material; receive the material request sent by the job seeker interview terminal, and the material request is used to request the digital human to listen to the material, and the material request is sent after the job seeker interview terminal collects the first sound signal of the target job seeker's response to the first interview question; according to the material request, obtain the digital human listening material, and generate the first transition frame material according to the first digital human interview material and the digital human listening material. The digital human listening material includes a silent digital human picture for the smooth transition of the digital human action between the first digital human interview material and the digital human listening material; send the first transition frame material and the digital human listening material to the job seeker interview terminal for the job seeker interview terminal to switch from the first digital human interview material to the digital human listening material through the first transition frame material after collecting the first sound signal, so that the digital human listens to the response of the target job seeker to the first interview question.

[0007] The embodiment of the present application provides an interview processing method based on a digital human, which is applied to a job seeker interview end, where a target interview room is provided for the job seeker interview end, and the target job seeker conducts an online interview based on a digital human for a target position through the target interview room; the method comprises:

[0008] During the online interview, the first digital human interview material for the target position issued by the digital human server is received, and the first digital human interview material includes picture data describing the digital human's movements and an audio signal describing the first interview question; the first digital human interview material is played, and during and after the first digital human interview material is played, the first sound signal of the target job seeker's answer to the first interview question is collected; if the first sound signal is collected, a material request for requesting the digital human to listen to the material is sent to the digital human server, and the digital human listening material includes a silent digital human picture; the first transition frame material and digital human listening material issued by the digital human server are received, and the first transition frame material is generated based on the first digital human interview material and the digital human listening material, and is used for the smooth transition of the digital human's movements in the first digital human interview material and the digital human listening material; the first digital human interview material is switched to the first transition frame material, and the digital human listening material continues to be played after the first transition frame material is played, so that the digital human listens to the target job seeker's answer to the first interview question.

[0009] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the above method.

[0010] An embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the above method.

[0011] An embodiment of the present application also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above method.

[0012] In the embodiment of the present application, during the online interview process, the introduction of listening materials can avoid the target interview room screen being still while the target job seeker is answering. Further, based on the first digital human interview material and the digital human listening material, a first transition frame material is generated. After playing the first digital human interview material, the first transition frame material can be switched to first, and then the first transition frame material can be switched to the digital human listening material. This can achieve a smooth transition of the digital human's actions in the first digital human interview material and the digital human listening material. In this way, not only can the smoothness of the online interview screen be improved, but also the richness and authenticity of the online interview interaction process can be improved, thereby enhancing the job seeker's interview experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0014] The embodiments and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0015] Figure 1 A schematic diagram of the structure of an interview processing system based on a digital human provided as an exemplary embodiment of the present application;

[0016] Figure 2a A schematic diagram of an application page provided for an exemplary embodiment of the present application;

[0017] Figure 2b A schematic diagram of a digital human interview process provided for an exemplary embodiment of the present application;

[0018] Figure 3 A flowchart of an interview processing method based on a digital human provided as an exemplary embodiment of the present application;

[0019] Figure 4 A flowchart of another digital human-based interview processing method provided for another exemplary embodiment of the present application;

[0020] Figure 5 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0023] In view of the above-mentioned technical problems, in an embodiment of the present application, during the online interview process, the introduction of listening materials can avoid the screen of the target interview room from being still while the target job seeker is answering. Furthermore, based on the first digital human interview material and the digital human listening material, a first transition frame material is generated. After playing the first digital human interview material, the first transition frame material can be switched to first, and then the first transition frame material can be switched to the digital human listening material. This can achieve a smooth transition of the digital human's actions in the first digital human interview material and the digital human listening material. In this way, not only can the smoothness of the online interview screen be improved, but also the richness and authenticity of the online interview interaction process can be improved, thereby enhancing the job seeker's interview experience.

[0024] A solution provided by an embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0025] Figure 1 Schematic diagram of an interview processing system based on a digital human provided by an exemplary embodiment of the present application. Figure 1 As shown, it includes: a job seeker interview terminal 101 and a digital human service terminal 102, and the job seeker interview terminal 101 and the digital human service terminal 102 can be connected for communication.

[0026] In the embodiment of the present application, the job seeker interview terminal 101 can be a terminal device used by any job seeker. The embodiment of the present application does not limit the implementation form of the job seeker interview terminal 101 used by any job seeker. For example, the job seeker interview terminal 101 can be a smart handheld device, such as a smart phone, a tablet computer, a laptop or a desktop computer, etc.; for another example, the job seeker interview terminal 101 can also be a smart wearable device, such as a smart watch, a smart bracelet, etc.; for another example, the job seeker interview terminal 101 can also be various smart home appliances with display screens, such as smart TVs, smart large screens or smart robots, etc. In addition, various applications can be installed on the job seeker interview terminal 101. The application can be an independent APP or a small program that depends on the operation of an independent APP. This embodiment does not limit this. The embodiment of the present application does not limit the type of application. For example, the application can be a shopping application, a taxi application, a video playback application, a recruitment application, and a digital person online interview application, etc.

[0027] It should be noted that this embodiment focuses on an application that can provide an interview room, through which job seekers and digital people can conduct online interviews based on digital people for the positions being recruited. In addition, an application can provide one or more interview rooms, and an interview room can interview multiple different positions, or an interview room can only interview one position, which is not limited in this embodiment of the application. For the sake of ease of description and distinction, any job seeker is referred to as a target job seeker, the job position of the target job seeker is referred to as a target position, and the interview room corresponding to the target position is referred to as a target interview room.

[0028] In the embodiment of the present application, there is no limitation on the implementation form of the digital human server 102. The digital human server 102 can be a physical server or a cloud server or a server cluster, etc.

[0029] In the embodiment of the present application, the communication connection between the above-mentioned job seeker interview terminal 101 and the digital human service terminal 102 can be a wireless communication connection or a wired communication connection. The specific communication connection method used can be determined by the implementation form of the digital human service terminal 102. When the implementation form of the digital human service terminal 102 is a physical machine server, both the wireless communication connection method and the wired communication connection method can be used. When the implementation form of the digital human service terminal 102 is a cloud server, the wireless communication connection method can be used. For the wireless communication connection method, optionally, the communication connection can be realized through a mobile network. Accordingly, the network standard of the mobile network can be 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), 5G, WiMax or any one of the new network standards that will appear in the future. In addition, when the job seeker interview terminal 101 and the digital human service terminal 102 are located in the same local area network, wireless communication connection can also be realized through Bluetooth, WiFi, infrared, zigbee or NFC.

[0030] In the embodiment of the present application, the digital human service end can generate multiple materials for the target position in advance, and store the multiple materials in a local designated location, so as to facilitate the use of the job seeker interview end when there is an online interview demand for the target position. The multiple materials can only include the digital human screen, and the multiple materials can also include the digital human screen and the audio signal at the same time. The digital human screen corresponds to the screen data describing the digital human action, and the screen data describing the digital human action includes the screen data describing the digital human mouth action and the screen data of the body action, and the audio corresponds to the audio signal describing the interview question. Among them, the screen data describing the digital human mouth action represents the change process of the digital human mouth action in the process of asking the interview question, and the screen data describing the digital human body action represents the change process of the digital human body action in the process of asking the interview question. In short, each digital human interview material includes the screen data describing the digital human action and the audio signal describing the interview question, and the screen data corresponds to the audio signal synchronously. The pre-generated multiple materials can be reused for the interview of the target position, thereby reducing the energy consumption of the online interview process based on digital human. In addition, in the process of online interview based on digital human, the required digital human materials can be matched from the pre-generated multiple materials and directly applied to the target interview room, without the need to generate digital human materials in real time during the interview process, and the matching process is simple, which can avoid the occurrence of digital human screen delay during the online interview process. For the specific implementation method of pre-generating multiple materials for the target position, please refer to the relevant description of the following embodiment, which will not be repeated here.

[0031] In the embodiment of the present application, the job seeker interview terminal is provided with a graphical user interface, each page of the target application can be displayed through the graphical user interface, and the target interview room corresponds to an access portal, which is set in the page of the target application. Figure 2a As shown, a page included in the target application is displayed on the page, in which an access entrance to the interview room is displayed. The access entrance is, for example, an access control corresponding to "Chat with the Agent", but the figure is only an exemplary illustration and does not constitute a limitation to the present application.

[0032] In the embodiment of the present application, in response to the target job seeker's triggering operation on the access entrance, the target interview room can be displayed, and the target job seeker then enters the interview room. At the same time, the target application will call the camera of the job seeker's interview end to collect the target job seeker's facial image, and send the facial image to the digital human server end, so that the digital human server end can identify the target job seeker's attribute information and perform subsequent processing based on the target job seeker's attribute information.

[0033] Accordingly, after receiving the facial image of the target job seeker, the digital human server can identify the attribute information of the target job seeker based on the facial image, and the attribute information includes but is not limited to age information and gender attributes, and the gender attributes include male and female. It should be noted that this embodiment does not limit the specific implementation method of identifying the attribute information of the target job seeker based on the facial image.

[0034] In an optional embodiment, based on the facial image, the attribute information of the target job seeker is identified, including: inputting the facial image into the feature recognition network layer of the face recognition model, identifying the facial feature information of the facial image, the feature information including facial features, texture features, and skin color features, etc., the facial features at least include the eyebrow feature information, eye feature information, nose feature information, mouth feature information, and ear feature information of the target job seeker, and the feature information specifically refers to the size, position, color, and shape of the above facial features; further, inputting the facial feature information into the age classification network layer, determining the age group of the target job seeker based on the facial feature information, so as to obtain the age of the target job seeker; and inputting the facial feature information into the gender analysis network layer, determining the gender attribute of the target job seeker based on the facial feature information. It should be noted that this embodiment does not limit the type of face recognition model. For example, the face recognition model can be an Insight Face model. InsightFace is a face recognition system based on deep learning, which has the characteristics of high efficiency, accuracy, and scalability. It can use advanced neural network technology to identify and verify faces from various complex environments.

[0035] Optionally, the facial image is input into the feature recognition network layer in the face recognition model to recognize facial features to obtain facial feature information, including: first identifying primary features of the facial image, such as edges and textures of facial features; gradually combining the primary features into more complex feature representations, which contain facial feature information; extracting features according to the feature hierarchy to be recognized to obtain facial feature information.

[0036] Optionally, the gender analysis network layer may include: a female face standard network layer, a male face standard network layer, and a similarity discrimination network layer; based on this, the facial feature information is input into the gender analysis network layer, and the gender attribute of the target job seeker is determined based on the facial feature information, including: presetting the first similarity threshold of the female face standard network layer and the second similarity threshold of the male face standard network layer; obtaining the first similarity between the low-dimensional feature vector of the facial features and the female face standard network layer and the second similarity between the low-dimensional feature vector of the facial features and the standard male face standard network layer through the similarity discrimination network layer; judging whether the first similarity exceeds the first threshold; if the first similarity exceeds the first threshold, it is preliminarily judged as a female; otherwise, it is preliminarily judged as a male; judging whether the second similarity exceeds the second threshold; if the second similarity exceeds the second threshold, it is preliminarily judged as a male; otherwise, it is preliminarily judged as a female; if the judgment is consistent, the judgment result is output; if the judgment is inconsistent, an unrecognizable result is output. The gender recognition process using a double verification method can improve the accuracy of the gender recognition result.

[0037] In an application scenario, the multiple materials include but are not limited to: multiple opening materials, and the opening materials are used to welcome the target job seeker to enter the live broadcast room. Each opening material includes an opening speech, and the opening speech refers to the speech that welcomes the target job seeker to enter the target live broadcast room. The opening speech may be related to the attribute information of the target job seeker and / or the time of entering the interview room. Based on this, after the job seeker terminal monitors the target job seeker entering the live broadcast room, it can send a material request for requesting the opening to the digital human terminal, and the material request includes the attribute information of the target job seeker and / or the time of entering the interview room. Correspondingly, the digital human terminal can select the opening that matches the attribute information of the target job seeker and / or the time of entering the interview room from multiple openings according to the attribute information of the target job seeker and / or the time of entering the interview room, and send it to the job seeker terminal for playback. In this way, the target job seeker can feel the enthusiasm of the interview room and enhance the user's interview experience.

[0038] For example, for a 25-year-old woman, the corresponding opening words may be "Good morning / afternoon", "Hello, beautiful", "Beauty, good morning / afternoon", etc.; for a 60-year-old woman, the corresponding opening words may be "Good morning / afternoon", "Hello, auntie", "Auntie, good morning / afternoon"; for a 25-year-old man, the corresponding opening words may be "Good morning / afternoon", "Hello, handsome guy", "Handsome guy, good morning / afternoon", etc.; for a 60-year-old man, the corresponding opening words may be "Good morning / afternoon", "Hello, uncle", "Uncle, good morning / afternoon", etc.

[0039] In one application scenario, the multiple materials also include: multiple digital human interview materials, which are used to ask questions to the target job seeker. Each digital human interview material contains an interview question, and each digital human interview material includes a digital human image and an audio signal. In addition, each digital human interview material may also contain an interview question.

[0040] During the online interview process, the digital human server can determine the first digital human interview material based on the attribute information of the target job applicant or the response information of the previous interview question, and the first digital human interview material includes the first interview question, and send the first digital human interview material to the job applicant interview end so that the job applicant interview end can play the first digital human interview material.

[0041] It should be noted that when the target job applicant just enters the interview room, the digital human server can determine the first digital human interview material based only on the target job applicant's attribute information; when the target job applicant has already answered other questions, the digital human server can determine the first digital human interview material based on the target job applicant's attribute information and / or the response information of the previous interview question.

[0042] In this embodiment, the first digital human interview material is determined based on the target job applicant's attribute information or the answer information to the previous interview question, including: matching among multiple materials based on the target job applicant's attribute information or the answer information to the previous interview question; if an interview material that matches the target job applicant's attribute information or the answer information to the previous interview question is matched, the matched interview material is used as the first digital human interview material; if an interview material that matches the target job applicant's attribute information or the answer information to the previous interview question is not matched, the first digital human interview material is generated based on the target job applicant's attribute information or the answer information to the previous interview question.

[0043] Furthermore, the digital human server stores multiple interview questions. Based on this, the first digital human interview material is generated according to the attribute information of the target job seeker or the answer information of the previous interview question, which can be achieved based on the following steps:

[0044] A1. Obtaining an image resource package of a digital human and a first frame rate of a first digital human interview material to be generated, wherein the image resource package includes multiple original base maps of the digital human, original facial region and original limb region images in each original base map, and position information of each original facial region and original limb region image in the corresponding base map.

[0045] Among them, the frame rate refers to the number of static images (frames) that can be displayed per second during video playback. The frame rate is an important indicator of video playback smoothness, which is usually measured in "frames per second" (FPS). For example, if the frame rate of the digital human to be generated is 25fps, 25 pictures will be played and explained within 1s. Based on this, the target audio feature information is segmented, including: obtaining the frame rate of the digital human video to be generated; segmenting the target feature information according to the duration of the audio data and the frame rate of the digital human video to be generated, and obtaining multiple audio feature fragments. When obtaining the frame rate of the digital human video to be generated, it is necessary to consider the purpose of the digital human video, the audience's viewing device, the video playback platform, and the required visual effects. For example, if the digital human video is used for online playback and needs to show smooth facial expressions and synchronized lip sync, a frame rate of 30fps or higher can be selected; if the digital human video is used in a movie or art project, 24fps can be selected to meet the viewing experience. In addition, technical limitations and costs need to be considered. Generally, a higher frame rate means that more data needs to be processed during editing and rendering, which may increase the demand and cost of computing resources. At the same time, the size of high-frame-rate digital human video files will also increase, which may affect storage and transmission efficiency. In practical applications, if applied to cloud intelligent digital human services, the video frame rate is generally not less than 25fps and not higher than 60fps. If applied to silicon-based digital human image cloning, the video frame rate is generally 30fps or 60fps, which can ensure the smoothness and detail of the video while taking into account the technical and cost limitations in practical applications.

[0046] The multiple original base images may include multiple groups of original digital human base images, each group of original digital human base images may be multiple continuous digital human images captured from any digital human video, and the mouth shapes and body shapes in the multiple continuous digital human images are continuously changing.

[0047] A2. Input the attribute information of the target job applicant or the answer information of the previous interview question into the question reasoning model, select the target question information that has a connection relationship with the attribute information of the target job applicant or the answer information of the previous interview question from multiple interview questions, and in the event of a matching failure, generate the target question information based on the first keyword of the attribute information of the target job applicant or the answer information of the previous interview question, and the target question information corresponds to the first digital human interview material.

[0048] A3. Input the target question information, the first frame rate, and multiple original base maps of the digital human, the original facial area and original limb area images in each original base map, and the position information of each original facial area and original limb area image in the corresponding base map into the interview material reasoning model based on the digital human, and convert the multiple mouth and limb deformation features into multiple pixel point data in the facial area and limb area and the position distribution information of each pixel point, so as to obtain the first digital human interview material.

[0049] Further optionally, the question reasoning model includes a semantic analysis network layer, a question matching network layer and a question generation network layer, then the above step A2: inputting the attribute information of the target job applicant or the answer information of the previous interview question into the question reasoning model, matching the target question information having a connection relationship with the attribute information of the target job applicant or the answer information of the previous interview question from multiple interview questions, and in the case of a matching failure, generating the target question information based on the first keyword of the attribute information of the target job applicant or the answer information of the previous interview question, the target question information corresponding to the first digital human interview material, can be implemented based on the following steps:

[0050] A21. Input the attribute information of the target job applicant into the semantic analysis network layer to extract the first key information of the attribute information; or, input the previous interview question and the answer information of the previous interview question into the semantic analysis extraction layer to extract the second key information of the previous interview question and the answer information of the previous interview question, and based on the second key information, analyze the third key information to be solved of the previous interview question.

[0051] A22, inputting the first key information into the problem matching network layer, and matching the target problem information matching the first key information from the multiple problem information; or, inputting the second key information into the problem matching network layer, and matching the target problem information matching the second key information from the multiple problem information;

[0052] A23. In case of a matching failure, the second key information is input into the question generation network layer, question information adapted to the second key information is generated, and the question information is used as the target question information.

[0053] Further optionally, the above step A3: the interview material reasoning model based on the digital human includes: an action mapping network layer and a feature conversion network layer, the target question information corresponds to audio data, audio duration and audio features; the target question information, the first frame rate and multiple original base maps of the digital human, the original face area and original limb area images in each original base map, and the position information of each original face area and original limb area image in the corresponding base map are input into the interview material reasoning model based on the digital human to obtain the first digital human interview material, which can be implemented based on the following steps:

[0054] A31. Input the audio duration, audio features and the first frame rate into the action mapping network layer, divide the audio features into multiple audio feature segments, infer the mouth shape changes and body changes based on the multiple audio feature segments, and generate the mouth and body deformation features corresponding to each audio segment. The mouth and body deformation features are multidimensional feature vectors, and the multidimensional feature vectors at least include mouth shape parameters and body parameters. Among them, the mouth shape parameters include: the opening and closing degree parameters of the mouth shape and the lip position parameters, and the body parameters include the opening and closing degree parameters of the body and the body position parameters.

[0055] A32. Input multiple mouth and limb deformation features, the first frame rate, multiple original base maps of the digital human, the original face area and original limb area images in each original base map, and the position information of each original face area and original limb area image in the corresponding base map into the feature conversion network layer; based on the mouth shape parameters and limb parameters corresponding to each mouth and limb deformation feature, convert each mouth and limb deformation feature into multiple pixel point data in the face area and limb area and the position distribution information of each pixel point, so as to obtain multiple target face area and limb area images.

[0056] A33, according to the position information of each original face region and limb region image in the corresponding original base map, the original face region and limb region images in the multiple original base maps are replaced with multiple target face and limb region images.

[0057] A34. According to the position information of each original face region and limb region image in the corresponding original base image, the original face region and limb region images in the multiple original base images are replaced with multiple target face region and limb region images to obtain multiple first digital human images with continuously changing mouth and limb shapes.

[0058] A35. Generate first digital human interview materials based on multiple first digital human images and audio data.

[0059] For example, for a domestic worker position, the first interview question is: "How much do you want to do as a domestic worker?" The domestic worker's answer is: "I want to find a job with a monthly salary of 8,000 yuan." In this regard, the second interview question that can be inferred can be: "What type of domestic worker do you want to find?"

[0060] Correspondingly, after receiving the first digital human interview material issued by the digital human server for the target position, the job seeker interview end plays the first digital human interview material, and collects the first sound signal of the target job seeker's answer to the first interview question during and after the first digital human interview material is played, so as to request the next digital human interview material from the digital human server based on the first sound signal. During and after the first digital human interview material is played, the target job seeker's sound signal can be collected and corresponding measures can be taken, which can avoid the job seeker from waiting and improve the user's interview experience.

[0061] In one application scenario, the multiple materials also include: digital human listening materials, which include silent digital human images. Silence means that there is no digital human audio signal. The digital human listening materials are used to play when the target job applicant is answering, which can avoid the target interview room's image being still during the target job applicant's answering process.

[0062] In this embodiment, if the job seeker terminal collects the first sound signal, it sends a material request for requesting the digital human to listen to materials to the digital human service end. The digital human listening materials include a silent digital human picture. After the job seeker interview end collects the first sound signal of the target job seeker's answer to the first interview question, it sends a material request for obtaining the digital human listening materials to the job seeker interview end. The material request is used to request the digital human to listen to materials.

[0063] Correspondingly, the digital human server receives the material request sent by the job seeker interview terminal, and obtains the digital human listening material according to the material request, so as to prepare to send the digital human listening material to the job seeker interview terminal for playback.

[0064] Furthermore, before the digital human listening material is sent to the job seeker interview end, a first transition frame material can be generated based on the first digital human interview material and the digital human listening material, for smooth transition of the digital human's movements between the first digital human interview material and the digital human listening material.

[0065] In this embodiment, a first transition frame material is generated based on a first digital human interview material and a digital human listening material, including: obtaining the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate of the second transition frame video to be generated; inputting the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate into the first digital human reasoning model, and analyzing the second deformation trend of the mouth and limbs between the last frame of the first digital human interview material and the first frame of the digital human listening material based on the mouth and limb features of the last frame of the first digital human interview material, the mouth and limb features of the first frame of the digital human listening material, and the second frame rate, and obtaining multiple second digital human images with continuously changing mouth and limb shapes based on the second deformation trend and the second frame rate; and generating the first transition frame material based on the multiple second digital human images and the second frame rate.

[0066] Optionally, the first digital human reasoning model includes: a feature analysis network layer, a feature matching network layer and an optical flow estimation network layer; the tail frame of the first questioning action video, the first frame of the target listening action video and the first frame rate are input into the first digital human reasoning model, and based on the mouth and limb features of the tail frame of the first questioning action, the mouth and limb features of the first frame of the target listening action video and the first frame rate, a first deformation trend of the mouth and limbs between the tail frame of the first questioning action video and the first frame of the target listening action video is analyzed, and based on the first deformation trend and the first frame rate, a plurality of second digital human images with continuously changing mouth and limb shapes are obtained, including: inputting the tail frame of the first questioning action video and the first frame of the target listening action video into the feature analysis network layer, extracting the first image features of the mouth and limb pixels of the tail frame of the first questioning action video and the second image features of the mouth and limbs of the first frame of the target listening action video; the first questioning action video The first image features of the pixels of the mouth and limb areas of the last frame of the target listening action video and the second image features of the mouth and limb areas of the first frame of the target listening action video are input into the feature matching network layer, and the first image features and the second image features are feature matched to obtain the correspondence between the same pixels of the mouth and limb areas in the two frames of images; the correspondence between the same pixels of the mouth and limb areas in the two frames of images is input into the optical flow estimation network layer, and the displacement between the same pixels is estimated to obtain the displacement results of the same pixels; based on the last frame of the first questioning action video, the first frame of the target listening action video and the displacement results of the same pixels of the mouth and limb areas in the two frames of images and the first frame rate, the number of second digital human images included in the first transition frame material to be generated and the positions of the pixels of the mouth and limb areas of each second digital human image are determined; based on the positions of the pixels of the mouth and limb areas of each second digital human image, multiple second digital human images are obtained.

[0067] The first digital human reasoning model and the second digital human reasoning model in the context may be based on a RIFE (Real-Time Intermediate Flow Estimation) video frame interpolation algorithm model.

[0068] Based on the RIFE video frame interpolation algorithm, the convolutional neural network is used to learn the optical flow field of two adjacent frames, estimate the motion information of pixels, and generate realistic transition frames. It usually involves the following key steps:

[0069] (1) Optical flow estimation: First, a convolutional neural network (such as FlowNet, RAFT, etc.) is used to learn the optical flow field between two adjacent frames in the video. The optical flow field describes the motion of each pixel in the image between consecutive frames. Specifically, the following steps are included: Using a convolutional neural network (CNN) to learn the optical flow field between two adjacent frames in a video usually involves the following key steps: Feature extraction: First, the network extracts features from the two input video frames. This is usually done through a convolutional layer, which can capture the local features of the image. Feature matching: Next, the network needs to find feature matches between the two frames. In networks such as FlowNetC, a special correlation layer is used to calculate the similarity between the two feature maps, thereby helping the network learn the correspondence between pixels. Optical flow estimation: The network estimates the optical flow based on the learned features and matching information. This process usually involves a prediction layer that outputs the motion vector of each pixel. Iterative optimization: An iterative optimization method is used to fine-tune the estimate of the optical flow field. RAFT uses a GRU-based recurrent update operator to retrieve values ​​from the multi-scale correlation volume and iteratively update the optical flow field. Upsampling and interpolation: To increase the resolution of the optical flow field, upsampling techniques are often used. For example, FlowNet uses a variational method instead of bilinear interpolation in the final stage to obtain a smooth and sub-pixel accurate optical flow field. End-to-end training: The entire network is trained end-to-end, meaning that from the input image pair to the output optical flow field, all layers of the network are optimized by backpropagation and gradient descent. Datasets: To train these networks, a large amount of training data is required. In addition to existing datasets such as KITTI and Sintel, researchers have also generated synthetic datasets such as Flying Chairs to provide more training samples. Loss function: During training, endpoint error (EPE) is usually used as the loss function, which calculates the Euclidean distance between the predicted optical flow and the true optical flow. Together, these steps enable convolutional neural networks to learn and predict complex motion patterns between two adjacent video frames.

[0070] (2) Feature extraction: The feature map is warped through the optical flow field so that the feature map of the previous frame is aligned to the current frame, which can better understand the movement of the pixels. Among them, warping the feature map through the optical flow field is a process of applying the estimated pixel motion information to image processing, which is very useful in many computer vision tasks, such as video interpolation and image registration. The following are the steps for warping the feature map: Calculate the optical flow field: First, a convolutional neural network (such as FlowNet, RAFT, etc.) is needed to calculate the optical flow field between two adjacent frames. This optical flow field contains the motion vector of each pixel in the x and y directions. Generate a grid: Create a grid of the same size as the input image, and each point in the grid represents the position of a pixel in the image. Apply optical flow displacement: Apply the motion vector in the optical flow field to the grid and calculate the new position corresponding to each point in another frame. Specifically, the optical flow vector is added to the coordinates of the original grid to obtain a distorted new grid. Feature map resampling: Use resampling techniques (such as bilinear interpolation, nearest neighbor interpolation, etc.) to extract the corresponding pixel values ​​from the original feature map according to the position of the new grid to generate a distorted feature map. Interpolation method: When resampling the feature map, you can choose different interpolation methods. For example, the 'bilinear' mode will perform linear interpolation between the four nearest pixels to obtain a smoother image; the 'nearest' mode directly selects the pixel value of the closest new position. This method has a small amount of calculation, but may produce a jagged effect. Handling boundary conditions: During the distortion process, some new positions may exceed the image boundary, and these situations need to be handled, such as by filling, cropping, etc. Generate a distorted image: Finally, generate a distorted image based on the resampled data. This image is the feature map with the optical flow field distortion applied.

[0071] (3) Transition frame generation: Using the estimated optical flow information and the data of the original frame, a neural network is used to generate transition frames. This step may involve a complex network architecture, such as U-Net, and a combination of multiple loss functions, including reconstruction loss, perceptual loss, smoothness loss, etc., to ensure that the generated frames are both natural and realistic.

[0072] It should be noted that in addition to generating the first transition frame material for the first digital human interview material and the digital human listening material, for any two adjacent materials played at the job applicant interview end, corresponding transition frame materials can be generated based on the last frame of the material played first and the first frame of the material played later, which is used for the smooth transition of the digital human's movements between adjacent played materials.

[0073] Furthermore, after receiving the first transition frame material and the digital human listening material sent by the digital human server, the job seeker terminal switches from the first digital human interview material to the first transition frame material while the target job seeker agrees, and continues to play the digital human listening material after the first transition frame material is played, so that the digital human listens to the target job seeker's answer to the first interview question.

[0074] During the process of the target job seeker answering, after the job seeker terminal detects the end of the first sound signal, it parses the first answer information of the first interview question from the first sound signal, and reports the first answer information to the digital human server to request the digital human server to send the second digital human interview material containing the second interview question. Further, the second transition frame material and the second digital human interview material sent by the digital human server are received, the digital human listening material is switched to the second transition frame material, and the second digital human interview material is played after the second transition frame material is played, so as to continue the online interview for the target position.

[0075] In this embodiment, after receiving the first response information for the first interview question reported by the job applicant interview end, the digital human server can match multiple materials based on the first response information; if the interview material is matched, the material is used as the second digital human interview material, and the second digital human interview material includes the second interview question; if the interview material is not matched, the second digital human reasoning model is used to generate the second digital human interview material based on the first response information; the second digital human interview material is sent to the job applicant interview end to continue the online interview with the target job applicant for the target position.

[0076] The technical solutions provided by the above-mentioned embodiments of the present application can avoid the screen of the target interview room from being still during the target job seeker's answering process by introducing the digital human listening material during the online interview process. Furthermore, a first transition frame material is generated based on the first digital human interview material and the digital human listening material. After playing the first digital human interview material, the first transition frame material can be switched to first, and then the first transition frame material can be switched to the digital human listening material. This can achieve a smooth transition of the digital human's actions in the first digital human interview material and the digital human listening material. In this way, not only can the smoothness of the online interview screen be improved, but also the richness and authenticity of the online interview interaction process can be improved, thereby enhancing the job seeker's interview experience.

[0077] The specific implementation process and related contents of some steps in the above embodiments are further elaborated in detail from other perspectives below.

[0078] In the embodiment of the present application, the digital human job seeker terminal can also match the first digital human interview material from multiple digital human interview materials according to the attribute information of the target job seeker. Optionally, matching the first digital human interview material from multiple digital human interview materials according to the attribute information of the target job seeker includes: determining the first digital human interview material based on the attribute information, the first digital human interview material including the first interview question; sending the first digital human interview material to the job seeker interview terminal, so that the job seeker interview terminal plays the first digital human interview material.

[0079] Correspondingly, the job seeker interview end receives and plays the opening material and / or the first digital human interview material issued by the digital human server for the target position, and collects the sound signal of the target job seeker's response to the first interview question during and after the playback of the first digital human interview material.

[0080] Furthermore, the multiple materials also include: multiple intermediate language materials. Each intermediate language material includes an intermediate language speech, which is used to provide feedback on the job seeker's response information. The intermediate language speech corresponding to different response information may be the same or different. Inserting intermediate language materials during the online interview process can avoid the screen from being static and increase the flexibility and authenticity of the interaction. For example, the intermediate language speech can be "OK, I understand", "You said it very well", etc.

[0081] In an embodiment of the present application, while the target job seeker is in the interview room, the job seeker interview end will also collect the target job seeker's voice signal. The job seeker interview end will also collect the target job seeker's voice signal, which can be implemented as follows: the target application calls the microphone of the job seeker interview end, and monitors whether the microphone obtains the target job seeker's response information during the call, and when the microphone is monitored to obtain the target job seeker's response information, the response information is sent to the digital human service end. This embodiment does not limit the implementation form of the target job seeker's response information. For example, the response information can be text information entered by the target job seeker, or an audio signal sent by the target job seeker, or other forms of information, etc. In actual applications, when the target job seeker starts to answer in the target interview room, the microphone can collect the target job seeker's response content in real time. In the case where the target job seeker's response information is an audio signal, the job seeker terminal can convert the audio signal into corresponding text information in real time.

[0082] In an optional embodiment, if the job applicant interview end collects the first sound signal, the first answer information of the first interview question is parsed from the first sound signal, and the silence time after the end of the first sound signal is monitored, when the silence time reaches the first time threshold, the first intermediate language material adapted to the first digital human interview material is played; during the process of playing the first intermediate language material, the first answer information is reported to the digital human server.

[0083] Correspondingly, the digital human service end receives the first response information reported by the job seeker interview end, matches the second digital human interview material from multiple digital human interview materials according to the first response information, and the second digital human interview material includes the second interview question; sends the second digital human interview material to the job seeker interview end to continue to conduct an online interview with the target job seeker for the target position.

[0084] Optionally, each digital human interview material has its own identification information, and the digital human interview material corresponding thereto can be matched according to any identification information. Based on this, matching the second digital human interview material from multiple digital human interview materials according to the first response information includes: inputting the first response information into the artificial intelligence matching model, determining the second keyword whose correlation with the first keyword is greater than a preset value based on the first keyword of the first response information, and matching the target digital human interview material from the digital human interview material library based on the identification information of the second keyword; and using the target digital human interview material as the second digital human interview material.

[0085] Optionally, sending the second digital human interview material to the job seeker interview end includes: using a transition frame inference model based on the last frame of the first intermediate language material and the first frame of the second digital human interview material to generate a second transition frame material, and sending the second transition frame material and the second digital human interview material to the job seeker interview end. The second transition frame material is used for the smooth transition of the digital human action between the first intermediate language material and the second digital human interview material.

[0086] Correspondingly, after receiving the second transition frame material and the second digital human interview material sent by the digital human server, the job seeker terminal switches from the first intermediate language material to the second transition frame material, and plays the second digital human interview material after the second transition frame material is played.

[0087] In another optional embodiment, if the job seeker terminal collects the first sound signal during the playback of the first digital person interview material, the first digital person interview material is stopped; or the first digital person interview material is muted. In other words, when the job seeker terminal is playing any digital person interview material and has not finished playing, the target job seeker can interrupt and answer at any time, and the digital person will "stop talking" accordingly, thereby improving the flexibility of interaction and enhancing the user's interview experience.

[0088] Following the above-mentioned embodiment in which the digital human server pre-generates multiple digital human interview materials for the target position, in an optional embodiment, multiple materials are pre-generated for the target position, including: obtaining a digital human image resource package, multiple speech scripts for the target position, the multiple speech scripts include but are not limited to historical opening words, historical interview questions and historical intermediary words, wherein the digital human image resource package contains multiple original digital human base maps adapted to the target position, original facial area and original limb area images in each digital human base map, and position information of each original facial area and original limb area image in the corresponding original digital human base map, and the historical opening words of the target position can be selected from the past interview sessions of the target interview room or the past interviews on site. The historical interview questions may be multiple interview questions collected from past interviews in the target interview room or past on-site interviews, etc. The historical intermediate words may be multiple intermediate words collected from past interviews in the target interview room or past on-site interviews, etc. Further, the multiple speech words are converted into audio signals respectively, and the audio signals correspond to audio feature information; further, each audio feature information is input into the digital human reasoning model to obtain multiple target digital human images with continuously changing mouth shapes and body shapes; further, based on each audio data and multiple target digital human images with continuously changing mouth shapes and body shapes, multiple digital human materials corresponding to the target positions are generated.

[0089] Optionally, multiple historical interview questions and multiple intermediate languages ​​are converted into audio signals respectively, including: extracting text information of each historical interview question and text information of each intermediate language respectively; using a large language model to analyze the grammatical and semantic structure of each text information, and performing prosody processing on each text to determine the stress, intonation and rhythm corresponding to each text; based on the grammatical and semantic structure of each text information and the stress, intonation and rhythm corresponding to each text, factor conversion is performed on each text to obtain the phonemes corresponding to each text, so as to obtain the factor sequence corresponding to each text, where the phoneme is the smallest unit in speech synthesis; and using a vocoder to convert the phoneme sequence into an audio signal to obtain an audio signal of each historical interview question and each intermediate language.

[0090] Further, optionally, after obtaining the audio signal of each speech, each audio signal can be converted into the target audio feature information required by the digital human reasoning model. The optional implementation method of converting each audio signal into the target audio feature information required by the digital human reasoning model is: extracting Fourier transform data from each audio signal; generating Mel spectrum data based on the Fourier transform data; and cropping, padding or resizing the Mel spectrum data according to the requirements of the digital human reasoning model for input data to obtain the target audio feature information corresponding to each audio signal, and each audio signal corresponds to its own audio feature information.

[0091] Optionally, each audio feature information is input into a digital human inference model to obtain multiple target digital human images with continuously changing mouth and limb shapes, including: mapping each audio feature into multiple mouth deformation features and multiple limb deformation features, and generating multiple target mouth area and limb area images with continuously changing mouth shapes and continuously changing limb shapes based on the multiple mouth deformation features and multiple limb deformation features; replacing the original face area and original limb area images in the multiple original digital human interviewer base maps with multiple target mouth area and limb area images according to the position information of each original face area and original limb area image in the corresponding original digital human interviewer base map to obtain multiple target digital human images with continuously changing mouth and limb shapes; generating multiple digital human materials corresponding to the target positions based on each audio signal and the multiple target digital human images with continuously changing mouth and limb shapes.

[0092] In another optional embodiment, multiple digital human materials are generated in advance for the target position, including: obtaining historical interview questions and intermediate language of the target position; converting the historical interview questions and intermediate language into audio signals; and recording multiple digital human interview materials based on the audio signals. The specific implementation of each step in this embodiment can be found in the relevant description of the previous optional embodiment, which will not be repeated here.

[0093] Figure 2b The process shown is an exemplary application of this application, which involves an online interview for a housekeeper. Figure 2b This scheme is described by way of example.

[0094] In this embodiment, the job seeker server is implemented as an APP, and the digital human server includes a business background part, a knowledge base part, and a digital human service part.

[0095] In an application scenario, after the interview begins, the job seeker terminal can call the camera to collect the face image of the target job seeker, and send the request containing the face image to the business background to request the opening material, and each opening has identification information. Correspondingly, the business background identifies the target job seeker's age, gender and other attribute information based on the face image, and sends the attribute information to the knowledge base. The knowledge base can use the model to match the opening that matches the attribute information. The opening has a corresponding id, and the id is sent to the digital human service through the business background to request the opening material corresponding to the id; accordingly, after the digital human service matches the opening material based on the id, the opening material is sent to the APP through the business background for playback.

[0096] In another application scenario, when the aunt is answering questions, the APP will collect the answering audio signal in real time and convert the answering audio information into text. After collecting the aunt's answer information, the text corresponding to the answer information will be sent to the knowledge base through the business background. The knowledge base uses the model to match the next question and its corresponding ID according to the answer information, and sends the next question and its corresponding ID to the digital human service through the business background to match the corresponding digital human interview material according to the next question and its corresponding ID, and send the material to the APP through the business background for playback. If the digital human service cannot match the corresponding digital human interview video according to the next question and its corresponding ID, it can generate the corresponding digital human interview material in real time according to the next question, and send the material to the APP for playback.

[0097] In another application scenario, when the APP is playing the digital human interview material, if the aunt's voice signal is collected, that is, the digital human is interrupted, the APP can collect the interruption information and send it to the knowledge base through the business background. The knowledge base can match the corresponding question and the id of the question according to the collected aunt's speech information, and send the question and the id of the question to the digital human service through the business background. The digital human service matches the corresponding digital human interview material according to the question and the id of the question, and sends the material to the APP through the business background for playback. If the digital human service cannot match the corresponding digital human interview video according to the next question and its corresponding id, it can generate the corresponding digital human interview material in real time according to the next question, and send the material to the APP for playback.

[0098] Figure 3 The following is a flowchart of a digital human-based interview processing method provided by an exemplary embodiment of the present application. Figure 3 As shown, the method is applied to the digital human server, and the target job seeker conducts an online interview based on a digital human for the target position in the target interview room provided by the digital human server. The method includes:

[0099] 301. During the online interview, determine a first digital person interview material according to the attribute information of the target job applicant or the answer information of the previous interview question, where the first digital person interview material includes a first interview question;

[0100] 302. Send the first digital person interview material to the job seeker interview terminal, so that the job seeker interview terminal plays the first digital person interview material;

[0101] 303. Receive a material request sent by the job seeker interview end, where the material request is used to request the digital human to listen to the material, and the material request is sent after the job seeker interview end collects a first sound signal of the target job seeker answering the first interview question;

[0102] 304. According to the material request, obtain the digital human listening material, and generate the first transition frame material according to the first digital human interview material and the digital human listening material, which is used for the smooth transition of the digital human action between the first digital human interview material and the digital human listening material. The digital human listening material includes a silent digital human picture.

[0103] 305. Send the first transition frame material and the digital human listening material to the job seeker interview end, so that the job seeker interview end switches from the first digital human interview material to the digital human listening material via the first transition frame material after collecting the first sound signal, so that the digital human listens to the target job seeker's answer to the first interview question.

[0104] In this embodiment, the digital human service end generates multiple materials for the target position in advance, and each interview material includes screen data describing the digital human's actions and audio signals describing the interview questions. Based on this, the first digital human interview material is determined according to the target job applicant's attribute information or the answer information of the previous interview question, including: matching among multiple materials according to the target job applicant's attribute information or the answer information of the previous interview question; if an interview material that matches the target job applicant's attribute information or the answer information of the previous interview question is matched, the matched interview material is used as the first digital human interview material; if an interview material that matches the target job applicant's attribute information or the answer information of the previous interview question is not matched, the first digital human interview material is generated according to the target job applicant's attribute information or the answer information of the previous interview question.

[0105] In this embodiment, the digital human server stores a plurality of interview questions; generating a first digital human interview material according to the attribute information of the target job applicant or the answer information of the previous interview question, including: obtaining an image resource package of the digital human and a first frame rate of the first digital human interview material to be generated, wherein the image resource package includes a plurality of original base maps of the digital human, original facial region and original limb region images in each original base map, and position information of each original facial region and original limb region image in the corresponding base map; inputting the attribute information of the target job applicant or the answer information of the previous interview question into the question reasoning model, and selecting from the plurality of interview questions a digital human having a connection with the attribute information of the target job applicant or the answer information of the previous interview question. The target question information of the target job applicant is matched with the target job applicant, and in case of a match failure, the target question information is generated based on the attribute information of the target job applicant or the first keyword of the answer information of the previous interview question, and the target question information corresponds to the first digital human interview material; the target question information, the first frame rate, and multiple original base maps of the digital human, the original face area and original limb area images in each original base map, and the position information of each original face area and original limb area image in the corresponding base map are input into the interview material reasoning model based on the digital human, and the multiple mouth and limb deformation features are respectively converted into multiple pixel point data in the face area and the limb area and the position distribution information of each pixel point, so as to obtain the first digital human interview material.

[0106] Optionally, the question reasoning model includes a semantic analysis network layer, a question matching network layer and a question generation network layer; the attribute information of the target job applicant or the answer information of the previous interview question is input into the question reasoning model, and the target question information having a connection relationship with the attribute information of the target job applicant or the answer information of the previous interview question is matched from multiple interview questions, and in the case of a matching failure, the target question information is generated based on the first keyword of the attribute information of the target job applicant or the answer information of the previous interview question, and the target question information corresponds to the first digital human interview material, including: inputting the attribute information of the target job applicant into the semantic analysis network layer, extracting the first key information of the attribute information; or matching the first key information of the previous interview question. The first key information is input into the question matching network layer, and the target question information matching the first key information is matched from multiple question information; or the second key information is input into the question matching network layer, and the target question information matching the second key information is matched from multiple question information; in case of matching failure, the second key information is input into the question generation network layer, and question information matching the second key information is generated, and the question information is used as the target question information.

[0107] Optionally, the interview material reasoning model based on digital humans includes: an action mapping network layer and a feature conversion network layer, the target question information corresponds to audio data, audio duration and audio features; the target question information, the first frame rate, and multiple original base maps of the digital human, the original facial area and original limb area images in each original base map, and the position information of each original facial area and original limb area image in the corresponding base map are input into the interview material reasoning model based on digital humans to obtain the first digital human interview material, including: inputting the audio duration, audio features and the first frame rate into the action mapping network layer, dividing the audio features into multiple audio feature segments, and reasoning about the mouth shape changes and limb changes based on the multiple audio feature segments to generate the mouth and limb deformation features corresponding to each audio segment, the mouth and limb deformation features are multidimensional feature vectors, and the multidimensional feature vectors at least include mouth shape parameters and limb parameters; multiple mouth and limb deformation features, the first frame rate, and multiple original base maps of the digital human, the original facial area and original limb area images in each original base map The face area and limb area images, and the position information of each original face area and limb area image in the corresponding base map are input into the feature conversion network layer; based on the mouth shape parameters and limb parameters corresponding to each mouth and limb deformation feature, each mouth and limb deformation feature is converted into multiple pixel point data in the face area and limb area and the position distribution information of each pixel point, so as to obtain multiple target face area and limb area images; according to the position information of each original face area and limb area image in the corresponding original base map, the original face area and limb area images in the multiple original base maps are replaced with multiple target face area and limb area images; according to the position information of each original face area and limb area image in the corresponding original base map, the original face area and limb area images in the multiple original base maps are replaced with multiple target face area and limb area images, so as to obtain multiple first digital human images with continuously changing mouth and limb shapes; according to the multiple first digital human images and audio data, the first digital human interview material is generated.

[0108] In this embodiment, a first transition frame material is generated according to a first digital human interview material and a digital human listening material, including: obtaining the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate of the second transition frame video to be generated; inputting the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate into the first digital human reasoning model, and analyzing the deformation trend of the mouth and limbs between the last frame of the first digital human interview material and the first frame of the digital human listening material based on the mouth and limb features of the last frame of the first digital human interview material, the mouth and limb features of the first frame of the digital human listening material, and the second frame rate, and obtaining a plurality of second digital human images with continuously changing mouth and limb shapes based on the deformation trend and the second frame rate; and generating the first transition frame material based on the plurality of second digital human images and the second frame rate.

[0109] Further optionally, first response information for the first interview question reported by the job applicant interview end is received, the first response information being parsed from the first sound signal by the job applicant interview end after the first sound signal ends; matching is performed among multiple materials according to the first response information; if the interview material is matched, the material is used as the second digital human interview material, and the second digital human interview material includes the second interview question; if the interview material is not matched, the second digital human interview material is generated based on the first response information using the second digital human reasoning model; the second digital human interview material is sent to the job applicant interview end to continue to conduct an online interview with the target job applicant for the target position.

[0110] The detailed implementation and beneficial effects of each module provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0111] Figure 4 The following is a flowchart of a digital human-based interview processing method provided by an exemplary embodiment of the present application. Figure 3 As shown, the method is applied to the job seeker interview end, where a target interview room is provided. The target job seeker and the digital human conduct an online interview based on the digital human for the target position through the target interview room. The method includes:

[0112] 401. Receive the first digital human interview material issued by the digital human server for the target position, where the first digital human interview material includes picture data describing the digital human's actions and an audio signal describing the first interview question;

[0113] 402. Play the first digital human interview material, and collect a first sound signal of the target job seeker answering the first interview question during and after the first digital human interview material is played;

[0114] 403. If the first sound signal is collected, a material request for requesting the digital human to listen to a material is sent to the digital human server, and the digital human listening material includes a silent digital human picture;

[0115] 404. Receive the first transition frame material and the digital human listening material sent by the digital human server, where the first transition frame material is generated based on the first digital human interview material and the digital human listening material, and is used for smooth transition of the digital human's movements between the first digital human interview material and the digital human listening material;

[0116] 405 . Switch from the first digital human interview material to the first transition frame material, and continue to play the digital human listening material after the first transition frame material is played, so that the digital human listens to the target job seeker's answer to the first interview question.

[0117] Further optionally, after monitoring the end of the first sound signal, the first answer information of the first interview question is parsed from the first sound signal, and the first answer information is reported to the digital human server to request the digital human server to send the second digital human interview material containing the second interview question; receive the second transition frame material and the second digital human interview material sent by the digital human server, the second transition frame material is generated based on the digital human listening material and the second digital human interview material, and is used for the smooth transition of the digital human's movements in the digital human listening material and the second digital human interview material; switch from the digital human listening material to the second transition frame material, and play the second digital human interview material after the second transition frame material is played, so as to continue the online interview for the target position.

[0118] The detailed implementation methods and beneficial effects provided by the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated in detail here.

[0119] Figure 5 The schematic diagram of the structure of the electronic device provided by the exemplary embodiment of the present application is shown in FIG. Figure 5 As shown, it includes: a memory 50a and a processor 50b; the memory 50a is used to store computer programs; the processor 50b is coupled to the memory 50a and is used to perform the following steps:

[0120] During the online interview, a first digital person interview material is determined according to the attribute information of the target job seeker or the answer information of the previous interview question, and the first digital person interview material includes the first interview question; the first digital person interview material is sent to the job seeker interview terminal, so that the job seeker interview terminal can play the first digital person interview material; a material request sent by the job seeker interview terminal is received, and the material request is used to request the digital person to listen to the material. The material request is sent after the job seeker interview terminal collects the first sound signal of the target job seeker answering the first interview question; according to the material request, the digital person listening material is obtained, and according to the first digital person interview material and the digital person listening material, a first transition frame material is generated, which is used for the smooth transition of the digital person's action between the first digital person interview material and the digital person listening material, and the digital person listening material includes a silent digital person picture; the first transition frame material and the digital person listening material are sent to the job seeker interview terminal, so that the job seeker interview terminal switches from the first digital person interview material to the digital person listening material through the first transition frame material after collecting the first sound signal, so that the digital person listens to the target job seeker's answer to the first interview question.

[0121] In this embodiment, the digital human service end generates multiple materials for the target position in advance, and each interview material includes screen data describing the digital human's actions and audio signals describing the interview questions. Based on this, the first digital human interview material is determined according to the target job applicant's attribute information or the answer information of the previous interview question, including: matching among multiple materials according to the target job applicant's attribute information or the answer information of the previous interview question; if an interview material that matches the target job applicant's attribute information or the answer information of the previous interview question is matched, the matched interview material is used as the first digital human interview material; if an interview material that matches the target job applicant's attribute information or the answer information of the previous interview question is not matched, the first digital human interview material is generated according to the target job applicant's attribute information or the answer information of the previous interview question.

[0122] In this embodiment, the digital human server stores a plurality of interview questions; when the processor 50b generates the first digital human interview material according to the attribute information of the target job applicant or the answer information of the previous interview question, it is specifically used to: obtain the image resource package of the digital human and the first frame rate of the first digital human interview material to be generated, the image resource package includes a plurality of original base maps of the digital human, the original face area and the original limb area images in each original base map, and the position information of each original face area and the original limb area image in the corresponding base map; input the attribute information of the target job applicant or the answer information of the previous interview question into the question reasoning model, and select the target job applicant's attribute information or the answer information of the previous interview question from the plurality of interview questions. The target question information has a connection relationship with the information, and in the case of matching failure, the target question information is generated based on the attribute information of the target job applicant or the first keyword of the answer information of the previous interview question, and the target question information corresponds to the first digital human interview material; the target question information, the first frame rate, and multiple original base maps of the digital human, the original face area and original limb area images in each original base map, and the position information of each original face area and original limb area image in the corresponding base map are input into the interview material reasoning model based on the digital human, and the multiple mouth and limb deformation features are respectively converted into multiple pixel point data in the face area and the limb area and the position distribution information of each pixel point to obtain the first digital human interview material.

[0123] Optionally, the question reasoning model includes a semantic analysis network layer, a question matching network layer and a question generation network layer; the processor 50b inputs the target job applicant's attribute information or the answer information of the previous interview question into the question reasoning model, matches the target question information that has a connection relationship with the target job applicant's attribute information or the answer information of the previous interview question from multiple interview questions, and generates the target question information based on the first keyword of the target job applicant's attribute information or the answer information of the previous interview question when the matching fails. When the target question information corresponds to the first digital human interview material, it is specifically used to: input the target job applicant's attribute information into the semantic analysis network layer to extract the first key information of the attribute information; or The previous interview question and the answer information of the previous interview question are input into the semantic analysis and extraction layer, the second key information of the previous interview question and the answer information of the previous interview question are extracted, and based on the second key information, the third key information to be solved of the previous interview question is analyzed; the first key information is input into the question matching network layer, and the target question information matching the first key information is matched from multiple question information; or, the second key information is input into the question matching network layer, and the target question information matching the second key information is matched from multiple question information; in case of matching failure, the second key information is input into the question generation network layer, question information adapted to the second key information is generated, and the question information is used as the target question information.

[0124] Optionally, the interview material reasoning model based on digital humans includes: an action mapping network layer and a feature conversion network layer, and the target question information corresponds to audio data, audio duration and audio features; when the processor 50b inputs the target question information, the first frame rate, and multiple original base maps of the digital human, the original face area and original limb area images in each original base map, and the position information of each original face area and original limb area image in the corresponding base map into the interview material reasoning model based on the digital human to obtain the first digital human interview material, it is specifically used to: input the audio duration, audio features and the first frame rate into the action mapping network layer, divide the audio features into multiple audio feature segments, and infer the mouth shape changes and limb changes based on the multiple audio feature segments to generate mouth and limb deformation features corresponding to each audio segment, the mouth and limb deformation features are multidimensional feature vectors, and the multidimensional feature vectors at least contain mouth shape parameters and limb parameters; multiple mouth and limb deformation features, the first frame rate, and multiple original base maps of the digital human, each original base map The original facial region and original limb region images in the corresponding base map and the position information of each original facial region and original limb region image in the corresponding base map are input into the feature conversion network layer, and based on the mouth shape parameters and limb parameters corresponding to each mouth and limb deformation feature, each mouth and limb deformation feature is converted into multiple pixel point data in the facial region and limb region and the position distribution information of each pixel point to obtain multiple target facial region and limb region images; according to the position information of each original facial region and limb region image in the corresponding original base map, the original facial region and limb region images in the multiple original base maps are replaced with multiple target facial region and limb region images; according to the position information of each original facial region and limb region image in the corresponding original base map, the original facial region and limb region images in the multiple original base maps are replaced with multiple target facial region and limb region images to obtain multiple first digital human images with continuously changing mouth and limb shapes; according to the multiple first digital human images and audio data, the first digital human interview material is generated.

[0125] In this embodiment, when the processor 50b generates the first transition frame material based on the first digital human interview material and the digital human listening material, it is specifically used to: obtain the last frame of the first digital human interview material, the first frame of the digital human listening material and the second frame rate of the second transition frame video to be generated; input the last frame of the first digital human interview material, the first frame of the digital human listening material and the second frame rate into the first digital human reasoning model, and analyze the deformation trend of the mouth and limbs between the last frame of the first digital human interview material and the first frame of the digital human listening material based on the mouth and limb features of the last frame of the first digital human interview material, the mouth and limb features of the first frame of the digital human listening material and the second frame rate, and obtain multiple second digital human images with continuously changing mouth and limb shapes based on the deformation trend and the second frame rate; generate the first transition frame material based on the multiple second digital human images and the second frame rate.

[0126] Further optionally, the processor 50b is also used to receive first response information for the first interview question reported by the job applicant interview end, the first response information being parsed from the first sound signal by the job applicant interview end after the first sound signal ends; matching is performed in multiple materials according to the first response information; if the interview material is matched, the material is used as the second digital human interview material, and the second digital human interview material includes the second interview question; if the interview material is not matched, generating the second digital human interview material based on the first response information using the second digital human reasoning model; and sending the second digital human interview material to the job applicant interview end to continue to conduct an online interview with the target job applicant for the target position.

[0127] Similarly, the processor 50b may also perform the following steps:

[0128] Receive the first digital human interview material issued by the digital human server for the target position, the first digital human interview material includes picture data describing the digital human's movements and an audio signal describing the first interview question; play the first digital human interview material, and collect the first sound signal of the target job seeker's answer to the first interview question during and after the first digital human interview material is played; if the first sound signal is collected, send a material request for requesting the digital human to listen to the material to the digital human server, the digital human listening material includes a silent digital human picture; receive the first transition frame material and digital human listening material issued by the digital human server, the first transition frame material is generated based on the first digital human interview material and the digital human listening material, and is used for the smooth transition of the digital human's movements between the first digital human interview material and the digital human listening material; switch from the first digital human interview material to the first transition frame material, and continue to play the digital human listening material after the first transition frame material is played, so that the digital human listens to the target job seeker's answer to the first interview question.

[0129] Further optionally, after monitoring the end of the first sound signal, the first answer information of the first interview question is parsed from the first sound signal, and the first answer information is reported to the digital human server to request the digital human server to send the second digital human interview material containing the second interview question; receive the second transition frame material and the second digital human interview material sent by the digital human server, the second transition frame material is generated based on the digital human listening material and the second digital human interview material, and is used for the smooth transition of the digital human's movements in the digital human listening material and the second digital human interview material; switch from the digital human listening material to the second transition frame material, and play the second digital human interview material after the second transition frame material is played, so as to continue the online interview for the target position.

[0130] The detailed implementation methods and beneficial effects provided by the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated in detail here.

[0131] Furthermore, if Figure 5 As shown, the electronic device also includes: a communication component 50c, a display 50d, a power component 50e, an audio component 50f and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0132] The exemplary embodiments of the present application further provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement the steps in the above-mentioned method embodiments.

[0133] An exemplary embodiment of the present application further provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the processor is enabled to implement the steps in the above-mentioned method embodiments.

[0134] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0135] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0136] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0137] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0138] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0139] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0140] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0142] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0143] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0144] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0145] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0146] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0147] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. An interview processing method based on digital human, characterized in that: Applied to the digital human server, the target job seeker conducts an online interview based on a digital human for the target position in the target interview room provided by the digital human server; the method includes: During the online interview, a first digital person interview material is determined based on the attribute information of the target job seeker or the answer information of the previous interview question, wherein the first digital person interview material includes a first interview question; Sending the first digital human interview material to the job seeker interview terminal, so that the job seeker interview terminal plays the first digital human interview material; Receiving a material request sent by the job seeker interview end, the material request is used to request a digital human to listen to the material, and the material request is sent after the job seeker interview end collects a first sound signal of the target job seeker answering the first interview question; According to the material request, the digital human listening material is acquired, and according to the first digital human interview material and the digital human listening material, a first transition frame material is generated for smooth transition of the digital human action between the first digital human interview material and the digital human listening material, wherein the digital human listening material includes a silent digital human picture; The first transition frame material and the digital human listening material are sent to the job seeker interview end, so that the job seeker interview end switches from the first digital human interview material to the digital human listening material via the first transition frame material after collecting the first sound signal, so that the digital human listens to the target job seeker's answer to the first interview question.

2. The method according to claim 1, characterized in that The digital human server generates multiple materials for the target position in advance, and each interview material includes picture data describing the digital human's actions and audio signals describing the interview questions; According to the attribute information of the target job seeker or the answer information of the previous interview question, the first digital human interview material is determined, including: Matching the multiple materials according to the attribute information of the target job seeker or the answer information of the previous interview question; If an interview material matching the attribute information of the target job seeker or the answer information of the previous interview question is matched, the matched interview material is used as the first digital human interview material; If no interview material matching the attribute information of the target job seeker or the answer information of the previous interview question is found, a first digital human interview material is generated according to the attribute information of the target job seeker or the answer information of the previous interview question.

3. The method according to claim 2, characterized in that The digital human server stores a plurality of interview questions; generating a first digital human interview material according to the attribute information of the target job seeker or the answer information of the previous interview question, including: Acquire an image resource package of the digital human and a first frame rate of a first digital human interview material to be generated, wherein the image resource package includes a plurality of original base maps of the digital human, original facial region and original limb region images in each original base map, and position information of each original facial region and original limb region image in a corresponding base map; Inputting the attribute information of the target job applicant or the answer information of the previous interview question into the question reasoning model, selecting the target question information having a connection relationship with the attribute information of the target job applicant or the answer information of the previous interview question from multiple interview questions, and generating the target question information based on the first keyword of the attribute information of the target job applicant or the answer information of the previous interview question when the matching fails, wherein the target question information corresponds to the first digital human interview material; The target question information, the first frame rate, and multiple original base maps of the digital human, the original facial area and original limb area images in each original base map, and the position information of each original facial area and original limb area image in the corresponding base map are input into the digital human-based interview material reasoning model, and the multiple mouth and limb deformation features are respectively converted into multiple pixel point data in the facial area and limb area and the position distribution information of each pixel point to obtain the first digital human interview material.

4. The method according to claim 3, characterized in that: The question reasoning model includes a semantic analysis network layer, a question matching network layer, and a question generation network layer; the attribute information of the target job applicant or the answer information of the previous interview question is input into the question reasoning model, and the target question information having a connection relationship with the attribute information of the target job applicant or the answer information of the previous interview question is matched from multiple interview questions, and in the case of a matching failure, the target question information is generated based on the first keyword of the attribute information of the target job applicant or the answer information of the previous interview question, and the target question information corresponds to the first digital human interview material, including: Input the attribute information of the target job seeker into the semantic analysis network layer, and extract the first key information of the attribute information; or input the previous interview question and the answer information of the previous interview question into the semantic analysis extraction layer, extract the second key information of the previous interview question and the answer information of the previous interview question, and analyze the third key information to be solved of the previous interview question based on the second key information; Input the first key information into the problem matching network layer, and match the target problem information matched with the first key information from the multiple problem information; or input the second key information into the problem matching network layer, and match the target problem information matched with the second key information from the multiple problem information; In the case of a matching failure, the second key information is input into the question generation network layer, question information adapted to the second key information is generated, and the question information is used as the target question information.

5. The method according to claim 4, characterized in that The digital human-based interview material reasoning model includes: an action mapping network layer and a feature conversion network layer, wherein the target question information corresponds to audio data, audio duration, and audio features; the target question information, the first frame rate, and multiple original base maps of the digital human, original facial regions and original limb region images in each original base map, and position information of each original facial region and original limb region image in the corresponding base map are input into the digital human-based interview material reasoning model to obtain the first digital human interview material, including: Inputting the audio duration, audio features and the first frame rate into the action mapping network layer, dividing the audio features into a plurality of audio feature segments, inferring mouth shape changes and body changes based on the plurality of audio feature segments, and generating mouth and body deformation features corresponding to each audio segment, wherein the mouth and body deformation features are multidimensional feature vectors, and the multidimensional feature vectors at least include mouth shape parameters and body parameters; Inputting the multiple mouth and limb deformation features, the first frame rate, the multiple original base maps of the digital human, the original face region and the original limb region images in each original base map, and the position information of each original face region and the original limb region image in the corresponding base map into the feature conversion network layer, and based on the mouth shape parameters and limb parameters corresponding to each mouth and limb deformation feature, converting each mouth and limb deformation feature into multiple pixel point data in the face region and the limb region and the position distribution information of each pixel point, to obtain multiple target face region and limb region images; According to the position information of each original face region and limb region image in the corresponding original base map, the original face region and limb region images in the multiple original base maps are replaced with the multiple target face region and limb region images; According to the position information of each original face region and limb region image in the corresponding original base image, the original face region and limb region images in the multiple original base images are replaced with the multiple target face region and limb region images to obtain multiple first digital human images with continuously changing mouth and limb shapes; First digital human interview material is generated according to the plurality of first digital human images and the audio data.

6. The method according to claim 1, characterized in that Generating a first transition frame material according to the first digital human interview material and the digital human listening material includes: Obtaining the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate of the second transition frame video to be generated; Inputting the last frame of the first digital human interview material, the first frame of the digital human listening material, and the second frame rate into the first digital human reasoning model, analyzing the deformation trend of the mouth and limbs between the last frame of the first digital human interview material and the first frame of the digital human listening material based on the mouth and limb features of the last frame of the first digital human interview material, the mouth and limb features of the first frame of the digital human listening material, and the second frame rate, and obtaining a plurality of second digital human images with continuously changing mouth and limb shapes based on the deformation trend and the second frame rate; Based on the plurality of second digital human images and the second frame rate, a first transition frame material is generated.

7. The method according to claim 2, characterized in that Also includes: receiving first answer information reported by the job seeker interview end to the first interview question, wherein the first answer information is parsed by the job seeker interview end from the first sound signal after the first sound signal ends; matching the multiple materials according to the first response information; If the interview material is matched, the material is used as the second digital person interview material, and the second digital person interview material includes the second interview question; If no interview material is matched, generating second digital human interview material based on the first response information using a second digital human reasoning model; The second digital human interview material is sent to the job seeker interview terminal to continue the online interview with the target job seeker for the target position.

8. An interview processing method based on digital human, characterized in that: Applied to a job seeker interview end, the job seeker interview end is provided with a target interview room, and the target job seeker conducts an online interview based on a digital person for a target position through the target interview room; the method includes: During the online interview, receiving the first digital human interview material issued by the digital human server for the target position, wherein the first digital human interview material includes picture data describing the digital human's actions and an audio signal describing the first interview question; Playing the first digital human interview material, and collecting a first sound signal of the target job seeker's answer to the first interview question during and after the playing of the first digital human interview material; If the first sound signal is collected, a material request for requesting the digital human to listen to a material is sent to the digital human server, and the digital human listening material includes a silent digital human picture; receiving a first transition frame material and a digital human listening material sent by the digital human server, wherein the first transition frame material is generated based on the first digital human interview material and the digital human listening material, and is used for smooth transition of digital human movements in the first digital human interview material and the digital human listening material; The first digital human interview material is switched to the first transition frame material, and after the first transition frame material is played, the digital human listening material continues to be played, so that the digital human listens to the target job seeker's response to the first interview question.

9. The method according to claim 8, characterized in that Also includes: After the monitoring of the first sound signal ends, the first answer information of the first interview question is parsed from the first sound signal, and the first answer information is reported to the digital human server to request the digital human server to send the second digital human interview material containing the second interview question; receiving a second transition frame material and a second digital human interview material sent by the digital human server, wherein the second transition frame material is generated based on the digital human listening material and the second digital human interview material, and is used for smooth transition of the digital human's movements between the digital human listening material and the second digital human interview material; Switch from the digital human listening material to the second transition frame material, and play the second digital human interview material after the second transition frame material is played, so as to continue the online interview for the target position.

10. An electronic device, characterized in that: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method of any one of claims 1-7 or 8-9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 7 or 8 to 9.

12. A computer program product, characterized in that The computer program product comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of any one of the methods of claims 1-7 or 8-9.

Images