Virtual object display method and related equipment

Abstract

The invention provides a virtual object display method and related equipment. The virtual object display method and the related equipment can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic and Internet of Vehicles. A virtual object generation page can be displayed, and the virtual object generation page comprises an object generation control; in response to a trigger operation for the object generation control, recording multiple segments of videos of the target object in different states to obtain an object video; when the target voice of the target object is received, a target virtual object corresponding to the target object is displayed, and the target virtual object comprises a virtual object generated based on the target voice, the follow-up video and the posture video. According to the embodiment of the invention, the trigger operation can be performed on the object generation control, and the multiple video segments of the target object in different states can be recorded, so that the convenience of recording the object video is improved, the target video of the target virtual object generated based on the target voice, the follow-up video and the posture video can be quickly displayed when the target voice is received, and the user experience is improved. The display efficiency of the virtual digital human is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence processing technology, and in particular to a virtual object display method and related equipment, wherein the related equipment includes a virtual object generation device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of AI technology, the audience of virtual digital people has expanded rapidly, and more and more users have begun to try to create their own virtual digital people. The creation of virtual digital people requires recording the user's video, and then using the recorded video to create virtual digital people.

[0003] Currently, the way to record user videos is through direct video recording, but this direct video recording method has strict requirements on the user's presentation skills, body movements and recording time. It is difficult to record a video that meets the production requirements of a virtual digital person in one go. Most users cannot record videos conveniently and efficiently, but need to record multiple times to obtain a video that meets the production requirements, which results in low display efficiency of virtual digital people.

[0004] In summary, there are currently problems with the poor convenience of video recording and the low efficiency of displaying virtual digital humans. Summary of the invention

[0005] The embodiments of the present application provide a virtual object display method and related equipment, which can improve the convenience of video recording, thereby improving the display efficiency of virtual digital people.

[0006] A virtual object display method, comprising:

[0007] Displaying a virtual object generation page, the virtual object generation page including an object generation control;

[0008] In response to a trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain an object video, wherein the object video includes a follow-up video and a posture video;

[0009] When the target voice of the target object is received, the target virtual object corresponding to the target object is displayed, and the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the gesture video.

[0010] Accordingly, an embodiment of the present application provides a virtual object generation device, including:

[0011] The acquisition unit may be used to display a virtual object generation page, wherein the virtual object generation page includes an object generation control;

[0012] A recording unit, which can be used to record multiple videos of the target object in different states in response to a trigger operation of the target object on the object generation control to obtain an object video, wherein the object video includes a follow-up video and a posture video;

[0013] The first display unit can be used to display a target virtual object corresponding to the target object when a target voice of the target object is received. The target virtual object includes a virtual object generated based on the target voice, the follow-up video and the gesture video.

[0014] Optionally, in some embodiments, the recording unit can be used to display a video recording page, which includes recording indication information and recording controls; in response to the target object's recording operation on the recording control, the video in each state corresponding to the recording indication information is recorded to obtain the object video.

[0015] Optionally, in some embodiments, the video recording page includes a text display control; the recording unit can be used to hide the target text and play the recording instruction information by voice in response to a trigger operation on the text display control when the video recording page includes the target text corresponding to the recording instruction information; when the video recording page does not include the target text corresponding to the recording instruction information, the target text is displayed in the video recording page in response to a trigger operation on the text display control.

[0016] Optionally, in some embodiments, the recording indication information indicates the recording order of each state; the recording unit can be used to display a recording guide page corresponding to a target recording mode, the target recording mode including a recording mode corresponding to a target state determined in the state based on the recording order; record a video of the target object in the target recording mode to obtain a video corresponding to the target recording mode; return to the step of displaying the recording guide page corresponding to the target recording mode until each state is the target state, obtain the video in each state, and fuse the videos in each state to obtain the object video.

[0017] Optionally, in some embodiments, the recording unit can be used to filter out a target state from the states based on a recording order; determine a target recording mode corresponding to the target object according to the target state; and display a recording guide page corresponding to the target recording mode.

[0018] Optionally, in some embodiments, the recording unit may be used to dynamically display recording countdown information for a target object; when the recording countdown information triggers a stop condition, a recording guide page corresponding to the target recording mode is displayed.

[0019] Optionally, in some embodiments, the recording unit can be used to display target follow-up guidance information and record a video of the target object following the target follow-up guidance information when the target recording mode is a follow-up recording mode, so as to obtain a video corresponding to the follow-up recording mode; when the target recording mode is a posture recording mode, display posture guidance information and record a video of the target object maintaining the target posture under the posture guidance information to obtain a video corresponding to the posture recording mode.

[0020] Optionally, in some embodiments, the recording unit can be used to obtain target follow-up guidance information corresponding to the target object; when the video recording page includes a target text corresponding to the recording instruction information, at least one follow-up text corresponding to the target follow-up guidance information is displayed in the recording guidance page, or the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played; when the video recording page does not include the target text corresponding to the recording instruction information, the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played in the recording guidance page.

[0021] Optionally, in some embodiments, the recording unit may be used to obtain recording progress information of the target object in the follow-up reading mode; and display progress prompt information corresponding to the recording progress information in the recording guide page.

[0022] Optionally, in some embodiments, the recording guidance page also includes a re-recording control; a recording unit can be used to record a video of the target object following the reading based on the target follow-up guidance information; in response to a trigger operation on the re-recording control, stop executing the step of recording the video of the target object following the reading based on the target follow-up guidance information; return to execute the step of displaying the target follow-up guidance information until the target follow-up guidance information is fully displayed, and obtain the video corresponding to the follow-up recording mode.

[0023] Optionally, in some embodiments, the display unit can be used to display a video recording prompt page, which includes recording prompt information corresponding to each recording mode and confirmation controls corresponding to the recording prompt information; in response to the target object's confirmation operation on the confirmation control, the video recording page is displayed.

[0024] Optionally, in some embodiments, the recording prompt information includes prompt text information and a prompt video thumbnail; the display unit can be used to respond to the target object's click operation on the prompt video thumbnail, display the prompt video corresponding to the prompt video thumbnail; and play the prompt video corresponding to the prompt video thumbnail.

[0025] Optionally, in some embodiments, the first display unit may also be used to send the object video to a server so that the server trains a preset object generation model based on the object video to obtain a virtual object generation model corresponding to the target object.

[0026] Optionally, in some embodiments, the first display unit can be specifically used to send the target voice of the target object to the server, so that the server uses the virtual object generation model to generate a target video of the target virtual object corresponding to the target object based on the target voice; receive the target video returned by the server, and play the target video to display the target virtual object.

[0027] The present application also provides another virtual object display method, including:

[0028] The object video of the target object sent by the receiving terminal includes a follow-up video and a posture video;

[0029] Based on the follow-up video and the posture video, the preset object generation model is trained to obtain the virtual object generation model corresponding to the target object;

[0030] When receiving a target voice of a target object sent by the terminal, a target video of a target virtual object of the target object is generated by using a virtual object generation model based on the target voice;

[0031] The target video is sent to the terminal so that the terminal plays the target video to display the target virtual object.

[0032] Accordingly, the present application provides another virtual object generation device, comprising:

[0033] A receiving unit, which can be used to receive an object video of a target object sent by a terminal, wherein the object video includes a follow-up video and a posture video;

[0034] The training unit can be used to train the preset object generation model based on the follow-up video and the posture video to obtain a virtual object generation model corresponding to the target object;

[0035] A generating unit may be configured to generate a target video of a target virtual object of the target object using a virtual object generating model based on the target voice when receiving the target voice of the target object sent by the terminal;

[0036] The second display unit can be used to send the target video to the terminal so that the terminal plays the target video to display the target virtual object.

[0037] Optionally, in some embodiments, the preset object generation model includes a preset vocal area generation model and a preset facial generation model; the training unit can be used to train the preset vocal area generation model based on the follow-up video to obtain the vocal area generation model; train the preset facial generation model according to the posture video to obtain the facial generation model; and use the vocal area generation model and the facial generation model as the virtual object generation model corresponding to the target object.

[0038] Optionally, in some embodiments, the follow-up video includes multiple follow-up video frames and audio frames corresponding to the follow-up video frames; the training unit can be used to identify the facial image of the target object in the follow-up video frame, and detect the vocalization area of ​​the target object in the facial image to obtain at least one vocalization area image; obtain at least one vocalization key point corresponding to the vocalization area image, and construct a vocalization area information set based on the vocalization key point; extract target audio data corresponding to the vocalization area in the audio frame, and train a preset vocalization area generation model based on the vocalization area information set and the target audio data to obtain a vocalization area generation model.

[0039] Optionally, in some embodiments, the generation unit can be used to predict the target vocal area image of the target object based on the target speech using a vocal area generation model; obtain the original virtual object video corresponding to the target object, and identify the target facial information of the original virtual object in the original virtual object video; based on the target facial information and the target vocal area image, use a facial generation model to perform facial correction on the target object to obtain a corrected facial image, and based on the corrected facial image, construct a target video of the target virtual object of the target object.

[0040] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement any virtual object display method provided in the embodiment of the present invention.

[0041] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for loading by a processor to execute any virtual object display method provided in the embodiment of the present application.

[0042] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which implements any virtual object display method provided in the embodiment of the present application when the computer program is executed by a processor.

[0043] The present application can display a virtual object generation page, which includes an object generation control; in response to the trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain object videos, which include follow-up videos and gesture videos; when the target voice of the target object is received, the target virtual object corresponding to the target object is displayed, and the target virtual object includes a virtual object generated based on the target voice, follow-up video and gesture video. Since the target object can record multiple videos of the target object in different states by triggering the object generation control in the virtual object generation page, the convenience of object video recording is improved, so that when the target voice is received, the target video of the target virtual object generated based on the target voice, follow-up video and gesture video can be quickly displayed to improve the display efficiency of the virtual digital person. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a scene schematic diagram of a virtual object display method provided in an embodiment of the present application;

[0046] Figure 2 is a flowchart of a method for displaying a virtual object provided in an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of a virtual object generation page provided in an embodiment of the present application;

[0048] Figure 4 is a schematic diagram of a video recording page provided in an embodiment of the present application;

[0049] Figure 5 is a second schematic diagram of a video recording page provided in an embodiment of the present application;

[0050] Figure 6 is a schematic diagram of an initial recording preview page provided in an embodiment of the present application;

[0051] Figure 7 3 are schematic diagrams of a video recording page provided in an embodiment of the present application;

[0052] Figure 8 is a schematic diagram of recording countdown information provided in an embodiment of the present application;

[0053] Fig. 9is a schematic diagram of a recording guide page provided in an embodiment of the present application;

[0054] Fig.10 It is a schematic diagram of recording reminder information provided by an embodiment of the present application;

[0055] Fig.11 is a schematic diagram of a posture guidance page provided in an embodiment of the present application;

[0056] Fig.12 is a schematic diagram of a recording interruption page provided in an embodiment of the present application;

[0057] Fig.13 is a schematic diagram of recording end indication information provided in an embodiment of the present application;

[0058] Fig.14 is a schematic diagram of a video saving page provided in an embodiment of the present application;

[0059] Fig.15 is a schematic diagram of a video confirmation page provided in an embodiment of the present application;

[0060] Fig.16 is a schematic diagram of return indication information provided in an embodiment of the present application;

[0061] Fig.17 is a schematic diagram of a submission page provided in an embodiment of the present application;

[0062] Fig.18 is a schematic diagram of a video recording prompt page provided in an embodiment of the present application;

[0063] Fig.19 FIG2 is a second flow chart of a method for displaying a virtual object provided in an embodiment of the present application;

[0064] Fig. 20 is a schematic diagram of a normalized facial image provided by an embodiment of the present application;

[0065] Fig.21 3 are schematic diagrams of the process of the virtual object display method provided in the embodiment of the present application;

[0066] Fig. 22 It is a schematic diagram of the model training provided in the embodiment of the present application;

[0067] Fig.23 is a schematic diagram of a target facial image provided by an embodiment of the present application;

[0068] Fig.24 is a schematic diagram of a label image provided by this application;

[0069] Fig.25 is a schematic diagram of a target video of a target virtual object generated by the present application;

[0070] Fig.26 is a schematic diagram of the structure of a virtual object generation device provided in an embodiment of the present application;

[0071] Fig. 27 FIG. 2 is a schematic diagram of the structure of the virtual object generation device provided in the embodiment of the present application.

[0072] Fig.28 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0074] The embodiment of the present application provides a virtual object display method and related equipment. The virtual object generation device can be integrated in a computer device, which can be a server or a terminal.

[0075] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, car terminal, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0076] For example, see Figure 1 Taking the virtual object generation device integrated in the terminal as an example, the terminal can display a virtual object generation page, which includes an object generation control; in response to the trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain object videos, which include follow-up videos and posture videos; when the target voice of the target object is received, the target virtual object corresponding to the target object is displayed, and the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the posture video.

[0077] Among them, the target virtual object may refer to a virtual digital person, and the video of the virtual digital person can be applied to multiple scenarios, such as AI virtual anchors, corporate virtual images, AI virtual assistants, etc. The virtual object display method provided in this application relates to artificial intelligence technology. This application can quickly record multiple videos of the target object in different states by triggering the object generation control in the virtual object generation page to obtain the object video, and then when the terminal receives the target voice, the target virtual object corresponding to the target object can be quickly displayed to improve the display efficiency of the virtual digital person.

[0078] Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0079] Among them, artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0080] It can be understood that in the specific implementation of the present application, the relevant data such as video, object video and target voice involved, when the following embodiments of the present application are applied to specific products or technologies, need to obtain permission or consent, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0081] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0082] This embodiment will be described from the perspective of a virtual object generation device, which can be specifically integrated in a computer device, which can be a server or a terminal. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, or other smart devices that can acquire data.

[0083] A method for generating a virtual object, comprising:

[0084] A virtual object generation page is displayed, the virtual object generation page includes an object generation control; in response to a trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain an object video, the object video includes a follow-up video and a posture video; when a target voice of the target object is received, a target virtual object corresponding to the target object is displayed, the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the posture video.

[0085] like Figure 2 As shown, the specific process of the virtual object display method is shown in steps S201 to S203:

[0086] S201: Display a virtual object generation page.

[0087] The virtual object generation page includes an object generation control. The object generation control may be displayed in the form of a virtual control, such as an icon.

[0088] It can be understood here that the present application can display a virtual object generation page to facilitate the generation of a target virtual object corresponding to the target object.

[0089] In step S201, there are multiple ways to "display the virtual object generation page". For example, the embodiment of the present application can display a virtual object generation operation page, which can include operation guide information for generating a virtual object and a target control corresponding to the operation guide information; in response to the target object's selection operation on the target control, the virtual object generation page is displayed. For example, when it is necessary to display the target virtual object of the target object, the target object can display the virtual object generation page by clicking the target control in the virtual object generation operation page.

[0090] For another example, in step S201, the method of "displaying the virtual object generation page" can also be: obtaining a trigger voice of the target object; in response to the trigger operation of the trigger voice, displaying the virtual object generation page. For example, when the target virtual object of the target object needs to be displayed, the target object makes a sound to form a trigger voice for triggering the display of the virtual object generation page; then, the computer device can recognize the trigger voice; when the computer device recognizes the trigger voice, the virtual object generation page can be displayed.

[0091] In an embodiment of the present application, before generating a target virtual object, object information of the target virtual object to be generated may also be entered. For example, the virtual object generation page may also include an object information entry control; in response to an input operation on the object information entry control, object information of the target virtual object to be generated is generated and the object information is displayed.

[0092] The object information may include, but is not limited to, an identifier of the target virtual object and an application scenario of the target virtual object.

[0093] In an embodiment of the present application, before generating a target virtual object, a shooting mode for the target object can also be selected. For example, the virtual object generation page can also include a mode selection control for the shooting mode; in response to the target object's selection operation on the mode selection control, the shooting mode of the target object is determined.

[0094] The shooting mode may include multiple shooting modes, for example, the shooting mode may include but is not limited to a segmented shooting mode, a direct recording shooting mode, and a video upload mode. The segmented shooting mode may refer to a mode of recording multiple videos of a target object in different states to obtain a video of the object; the direct recording shooting mode may refer to a mode of recording a video of the target object at one time; and the video upload mode may refer to a mode of uploading a video of the target object that has been shot locally.

[0095] Among them, the mode selection control may include a segmented selection control corresponding to the segmented shooting mode, a direct recording selection control corresponding to the direct recording shooting mode, and an upload selection control corresponding to the video upload mode.

[0096] Among them, the method of the step of "determining the shooting mode of the target object in response to the target object's selection operation on the mode selection control" can be specifically: in response to the target object's selection operation on the segmented selection control, determining that the shooting mode of the target object is the segmented shooting mode.

[0097] Here we use a specific example to illustrate. Figure 3As shown, the virtual object generation page 31 of the present application may include an object information entry control 311, a segmented selection control 312 corresponding to the segmented shooting mode, a direct recording selection control 313 corresponding to the direct recording shooting mode, an upload selection control 314 corresponding to the video upload mode, and an object generation control 315. The target object can click the object information entry control 311 to enter the object information of the target virtual object to be generated. The target object can click the segmented selection control 312 of the segmented shooting mode to determine that the shooting mode of the target object is the segmented shooting mode; then, the target object can trigger the object generation control 315, and then the computer device can execute step S202.

[0098] S202: In response to a triggering operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain an object video.

[0099] The state may include multiple states, for example, the state may include but is not limited to a follow-up state, a silent state, a standing state, and a hand-raising state. Each state may correspond to at least one video. For example, when the target object is in a follow-up state, one or two videos of the target object in the follow-up state may be recorded.

[0100] The object video includes a follow-up video and a posture video. The follow-up video may refer to a video of the target object in a follow-up state; the posture video may refer to a video of the target object in a specified posture state, and the specified posture state may include but is not limited to any one of a silent state, a standing state, and a hand-raising state, or a combination of at least two states.

[0101] In step S202, the method of the step "recording multiple videos of the target object in different states to obtain the object video" can be: displaying a video recording page, the video recording page including recording instruction information and recording controls; in response to the target object's recording operation on the recording control, recording the video in each state corresponding to the recording instruction information to obtain the object video.

[0102] The recording instruction information may refer to information for indicating that the target object is recorded in each state. For example, the recording instruction information may indicate the order of recording each state. For example, if the state includes a follow-up state and a silent state, the recording instruction information may indicate that the follow-up state is recorded first, and then the silent state is recorded.

[0103] The recording control may be a virtual control. Specifically, the recording control may be presented in a virtual icon of a preset shape, such as a circular virtual icon.

[0104] For example, the target object, such as a user, can click on a recording control in a video recording page, so that the computer device can respond to the click operation, record the video in each state corresponding to the recording indication information, and obtain the object video.

[0105] In the present application, there are two ways to display the recording instruction information in the video recording page, for example, it can be displayed in the form of voice playback or in the form of text. The target object can choose the form of text display according to specific needs. For example, after the step "displaying the video recording page", when the video recording page includes the target text corresponding to the recording instruction information, in response to the trigger operation of the text display control, the target text is hidden and the recording instruction information is played by voice; when the video recording page does not include the target text corresponding to the recording instruction information, in response to the trigger operation of the text display control, the target text is displayed in the video recording page.

[0106] The text display control may be a virtual control. Specifically, the text display control may be displayed as a text mark. For example, the text display control may be displayed as a “word”.

[0107] It is understood that the present application can set the recording instruction information to be displayed in the form of voice playback as the default display form. Then, in the case where the video recording page includes the target text corresponding to the recording instruction information, for example, Figure 4 As shown, when the target object performs a trigger operation on the text display control 41, the computer device hides the target text 42 in response to the trigger operation on the text display control, and maintains the voice playback recording instruction information.

[0108] For example, Figure 5 As shown, in the case where the video recording page does not include the target text corresponding to the recording indication information, when the target object triggers the text display control 41, the computer device responds to the trigger operation on the text display control, displays the target text in the video recording page, and maintains the voice playback recording indication information.

[0109] Specifically, in step S201, the step "displaying the video recording page" can be specifically: displaying the recording preview page, the recording preview page including the preview screen, shooting reminder information, and the target confirmation control corresponding to the shooting reminder information; in response to the confirmation operation on the target confirmation control, displaying the video recording page.

[0110] Among them, the shooting reminder information can be displayed above the preview screen; when the shooting reminder information is displayed above the preview screen, the brightness of the preview screen becomes dim.

[0111] The step of "displaying the recording preview page" may be performed as follows: when the computer device does not start the camera, an initial recording preview page is displayed, the initial recording preview page includes camera shutdown information and a startup setting control for the camera; in response to a selection operation on the startup setting control, the camera is started and a video recording page is displayed. When the computer device starts the camera, the recording preview screen is directly displayed.

[0112] Specifically, for example, when the computer device does not start the camera, the following Figure 6 The initial recording preview page shown in FIG. 1 includes camera shutdown information 61 and a camera startup setting control 62. The target object can select the startup setting control, and the computer device can start the camera and display the following information: Figure 7 Video recording page.

[0113] In such Figure 6 The initial recording preview page shown may also include a candidate confirmation control 63; when the target object clicks the candidate confirmation control 63, the computer device may hide the camera closing information in response to the click operation on the candidate confirmation control.

[0114] In such Figure 6 The initial preview page may also include a conversion control 64 for the camera; when the camera is a rear-facing camera, the target object may trigger the conversion control 64, and the computer device may respond to the triggering operation on the conversion control to switch the camera to a front-facing camera; when the camera is a front-facing camera, the target object may click the conversion control, and the computer device may respond to the clicking operation on the conversion control to switch the camera to a rear-facing camera. It should be noted that the conversion control 64 may also be displayed in the video recording page, such as Figure 4 , Figure 5 shown.

[0115] for Figure 7 , Figure 7 In the video recording page shown in , volume prompt information 71 and an information confirmation control 72 corresponding to the volume prompt information can also be displayed; in response to the confirmation operation of the information confirmation control 72, the volume prompt information 71 is hidden, and the video recording page is kept displayed. Figure 7 After displaying, you can display the following Figure 4 or Figure 5 page.

[0116] After the video recording page is displayed, when the target object performs a recording operation on the recording control 43 in the video recording page, the video in each state corresponding to the recording indication information can be recorded to obtain the target video. Specifically, the method of "recording the video in each state corresponding to the recording indication information to obtain the target video" can be as shown in steps S1 to S3:

[0117] S1. Display the recording guide page corresponding to the target recording mode.

[0118] The recording instruction information indicates the recording order of each state. The target recording mode includes a recording mode corresponding to the target state determined in the state based on the recording order.

[0119] Each recording mode corresponds to a state, and each recording mode may also correspond to multiple states. For example, when the state includes a follow-up reading state and a silent state, the recording mode may correspond to one state such as the follow-up reading state, or the recording mode may correspond to multiple states such as the follow-up reading state and the silent state.

[0120] Among them, the recording mode may include at least one recording mode. When the recording mode includes two recording modes, for example, the recording mode includes a first recording mode and a second recording mode, and the state includes a follow-up state and a standing state. The follow-up state is the target state corresponding to the first recording mode, and the standing state is the target state corresponding to the second recording mode. The recording order is to first record the video corresponding to the follow-up state, and then record the video corresponding to the standing state. Correspondingly, this application preferentially displays the recording guide page corresponding to the first recording mode to record the video corresponding to the follow-up state, and then displays the recording guide page corresponding to the second recording mode to record the video corresponding to the standing state.

[0121] In step S1, the step of "displaying the recording guide page corresponding to the target recording mode" can be: based on the recording order, filtering out the target state in the state; determining the target recording mode corresponding to the target object according to the target state; and displaying the recording guide page corresponding to the target recording mode.

[0122] Among them, after the target state is filtered out based on the recording order, the target recording mode can be determined. Specifically, the step of "determining the target recording mode corresponding to the target object according to the target state" can be: obtaining a mapping relationship set, the mapping relationship set including the mapping relationship between the preset state and the preset target recording mode; determining the target recording mode corresponding to the target state according to the target state and the mapping relationship.

[0123] After displaying the recording guide page corresponding to the target recording mode, the present application can dynamically display the recording countdown information for the target object. For example, the recording countdown information can be a countdown of 3 seconds, a countdown of 5 seconds, and so on. Specifically, the recording countdown information 81 can be as follows: Figure 8 shown.

[0124] Specifically, when dynamically displaying the recording countdown information for the target object, the present application can dim the brightness of the recording guide page and dynamically display the recording countdown information on the dimmed recording guide page.

[0125] Correspondingly, when the recording countdown information triggers the stop condition, the recording guide page corresponding to the target recording mode is displayed. Specifically, when the recording countdown information triggers the stop condition, the recording countdown can be hidden, and then the recording guide page corresponding to the target recording mode is displayed.

[0126] The stop condition may be the completion of displaying the countdown information, playing music, displaying pictures, and so on.

[0127] S2. Record a video of the target object in a target recording mode to obtain a video corresponding to the target recording mode.

[0128] The video in the target recording mode may include at least one video. When the video in the target recording mode includes multiple initial videos, the multiple initial target videos may be spliced ​​to obtain a video corresponding to the target recording mode.

[0129] In this application, there can be multiple target recording modes. Here, the target recording modes including the follow-up recording mode and the gesture recording mode are used as examples for explanation. Correspondingly, in step S2, there are two ways of "recording the video of the target object in the target recording mode to obtain the video corresponding to the target recording mode", such as the following method 1 and method 2:

[0130] Among them, the follow-up recording mode may refer to the recording mode corresponding to the follow-up state, and the posture recording mode may refer to the recording mode corresponding to the posture state, wherein the posture state refers to the non-follow-up state, for example, the posture state may include but is not limited to at least one of the silent state, standing state and hand-raising state.

[0131] Method 1: When the target recording mode is the follow-up recording mode, in this case, the method of "recording the video of the target object in the target recording mode to obtain the video corresponding to the target recording mode" can be: displaying the target follow-up guidance information, and recording the video of the target object following the target follow-up guidance information to obtain the video corresponding to the follow-up recording mode.

[0132] The target follow-up guidance information may refer to information for guiding the target object to speak. Specifically, the target follow-up guidance information may be represented as a sentence, a paragraph, or an article, etc. The display form of the target follow-up guidance information may include but is not limited to at least one of a text display form and a voice playback display form. The target follow-up guidance information may also be represented as a question. For example, through the question, the target object may be guided to answer the question or the target object may be guided to follow the question.

[0133] The target follow-up guidance information may be information input by the target object. For example, before displaying the recording guidance page, the present application may also include a target input control, such as a virtual object generation page; in response to an input operation on the target input control, the target follow-up guidance information is generated. The input operation on the target input control may be voice input, text input, and so on.

[0134] For example, when the target follow-up guidance information is a paragraph or an article, in the process of recording a video of the target object following the target follow-up guidance information, the computer device can first display a sentence, such as the first sentence, and then the target object follows the first sentence; then, the computer device displays another sentence, such as the second sentence, and then the target object follows the second sentence, and so on. Among them, the application can set the follow-up time of the target object to be consistent with the display time of the follow-up words. For example, the follow-up words are the third sentence in the article, and the display time of the third sentence is 8 seconds. Then, correspondingly, the follow-up time of the target object can also be set to 8 seconds.

[0135] Of course, the present application may also not limit the target object's follow-up reading time. For example, the follow-up reading is the fourth sentence in the article, and the display time of the fourth sentence is 10 seconds. When the fourth sentence is displayed, the target object can follow the fourth sentence, and the time when the target object speaks is the target object's follow-up reading time. When the computer device does not detect the target object's voice, the next sentence in the article, such as the fifth sentence, can be displayed.

[0136] When the display form of the target follow-up guidance information includes a text display form and a voice playback display form, in method 1, the method of "displaying the target follow-up guidance information" can be: obtaining the target follow-up guidance information corresponding to the target object; after obtaining the target follow-up guidance information corresponding to the target object, the target follow-up guidance information can be displayed. For example, there are multiple situations, such as situation (1) and situation (2):

[0137] Case (1): When the video recording page includes a target text corresponding to the recording instruction information, at least one follow-up text corresponding to the target follow-up guide information is displayed in the recording guide page, or the text voice of at least one follow-up text corresponding to the target follow-up guide information is played.

[0138] It can be understood here that before the recording guide page is displayed, when the video recording page does not include the target text corresponding to the recording instruction information, if the target object triggers the text display control, accordingly, the video recording page will display the target text corresponding to the recording instruction information. In this way, when the recording guide page is displayed, the display form of the target follow-up guidance information is a text display form, for example, at least one follow-up text corresponding to the target follow-up guidance information is displayed; or, the display form of the target follow-up guidance information can also be a voice playback display form, for example, the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played. Of course, in an embodiment of the present application, the target follow-up guidance information can be displayed in a voice playback display form and a text display form at the same time. For example, at least one follow-up text corresponding to the target follow-up guidance information can be displayed in the recording guide page at the same time, and the text voice of at least one follow-up text corresponding to the target follow-up guidance information can be played.

[0139] Case (2): When the video recording page does not include the target text corresponding to the recording instruction information, the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played in the recording guide page.

[0140] It can be understood here that before the recording guide page is displayed, in the case where the video recording page includes the target text corresponding to the recording instruction information, if the target object triggers the text display control, then, accordingly, the video recording page does not include the target text corresponding to the recording instruction information. In this way, when the recording guide page is displayed, the display form of the target follow-up guide information is the voice playback display form.

[0141] After the present application displays the target follow-up guidance information, it can also display the progress prompt information in the follow-up mode in the recording guide page. Specifically, the method of displaying the progress prompt information in the follow-up mode in the recording guide page can be: obtaining the recording progress information of the target object in the follow-up mode; and displaying the progress prompt information corresponding to the recording progress information in the recording guide page.

[0142] The method of “obtaining the recording progress information of the target object in the follow-up reading mode” may be: obtaining the display time of the follow-up reading text in the follow-up reading mode; and determining the recording progress information of the target object in the follow-up reading mode according to the display time.

[0143] Wherein, when the follow-reading guidance information includes the follow-reading guidance text, the follow-reading text may refer to a text segment in the follow-reading guidance text. For example, when the follow-reading guidance text is a paragraph of text, the follow-reading text may be each sentence in the text.

[0144] The display time may refer to the time accumulated from the time when the follow-up guidance text is first displayed; for example, the display time may refer to the time from the time when the first follow-up text is displayed to the time when the current follow-up text is displayed.

[0145] Among them, the present application can use the display time as the recording progress information of the target object in the follow-up reading mode. Alternatively, the present application can obtain the follow-up reading time of the target object, then add the follow-up reading time and the display time to obtain the added time; and use the added time as the recording progress information of the target object in the follow-up reading mode.

[0146] Among them, the method of "displaying progress prompt information corresponding to the recording progress information in the recording guide page" can be: obtaining progress prompt data corresponding to the progress information; and drawing the progress prompt information corresponding to the recording progress information in the recording guide page according to the progress prompt data to display the progress prompt information.

[0147] The progress prompt information may be represented as a progress bar.

[0148] This is explained with a specific example. For example, when the video recording page includes target text corresponding to the recording instruction information, such as Fig. 9 As shown, at least one follow-up text 91 corresponding to the target follow-up guidance information is displayed in the recording guide page, and the text voice corresponding to the follow-up text 91 is played. In this process, the application can obtain the recording progress information of the target object in the follow-up mode, and display the progress prompt information 92 corresponding to the recording progress information in the recording guide page. The progress prompt information 92 may include a progress bar, video name and recording time. Fig. 9 As shown, the object preview screen of the target object can be displayed in the recording guide page, and the follow-up text and progress prompt information of this application are displayed above the object preview screen.

[0149] When the target object is recording in the follow-up recording mode, the target object's video recording may be interrupted by poor network or incoming calls, or the target object may interrupt the video recording. In this case, you can re-record the video in the follow-up recording mode.

[0150] Specifically, the recording guidance page also includes a re-recording control; in this case, the method of "recording a video of the target object following the target follow-reading guidance information to obtain a video corresponding to the follow-reading recording mode" can be: recording a video of the target object following the target follow-reading guidance information; in response to a trigger operation on the re-recording control, stopping the step of recording the video of the target object following the target follow-reading guidance information; returning to the step of displaying the target follow-reading guidance information until the target follow-reading guidance information is fully displayed, to obtain the video corresponding to the follow-reading recording mode.

[0151] For example, Fig. 9 As shown, the object preview screen of the target object can be displayed in the recording guide page, and the re-recording control 93 of the present application can be displayed above the object preview screen. In the process of recording the video of the target object following the reading based on the target follow-up guidance information, when the target object clicks the re-recording control 93, the computer device responds to the triggering operation on the re-recording control and stops executing the step of recording the video of the target object following the reading based on the target follow-up guidance information. Then, the computer device re-records the video corresponding to the follow-up recording mode.

[0152] Method 2: When the target recording mode is the posture recording mode, in this case, the method of "recording the video of the target object in the target recording mode and obtaining the video corresponding to the target recording mode" can be: displaying the posture guidance information, and recording the video of the target object maintaining the target posture under the posture guidance information, and obtaining the video corresponding to the posture recording mode.

[0153] The posture guidance information may refer to information for guiding the target object to maintain a specified posture for video recording. Specifically, the posture guidance information may be information for guiding the target object to remain silent and maintain a specified posture. The specified posture may be, for example, a speech posture, or a hand-raising posture, etc.

[0154] There are many ways to "display posture guidance information": for example, obtaining posture guidance information; directly broadcasting posture guidance information by voice. For another example, obtaining posture guidance information; when the video recording page includes the target text corresponding to the recording instruction information, displaying the posture guidance text corresponding to the posture guidance information in the recording guidance page, or playing the target text voice corresponding to the posture guidance text corresponding to the posture guidance information. When the video recording page does not include the target text corresponding to the recording instruction information, playing the target text voice corresponding to the posture guidance text corresponding to the posture guidance information in the recording guidance page.

[0155] In the embodiment of the present application, the posture guidance information may be set before the recording guidance page is displayed. For example, when the video recording page is displayed, the video recording page may also include a posture setting control; in response to a selection operation on the posture setting control, a specified posture of the target object is determined; and posture guidance information corresponding to the specified posture is obtained.

[0156] There are many ways to "obtain posture guidance information corresponding to a specified posture", for example, the video recording page may also include an information input control corresponding to a specified posture; in response to an input operation on the information input control, posture guidance information corresponding to the specified posture is generated. For another example, the posture guidance information is pre-set information, based on which, the present application may obtain posture guidance information corresponding to the specified posture based on the posture identifier of the specified posture.

[0157] Of course, after displaying the posture guidance information, the present application can also display the target progress prompt information in the posture recording mode in the recording guidance page. Specifically, the method for the present application to display the target progress prompt information in the posture recording mode in the recording guidance page can be: obtaining the target recording progress information of the target object in the posture recording mode; and displaying the target progress prompt information corresponding to the target recording progress information in the recording guidance page.

[0158] The method of “obtaining target recording progress information of the target object in the gesture recording mode” may be: obtaining the recording time in the gesture recording mode; and using the recording time as the target recording progress information of the target object in the gesture recording mode.

[0159] The method of "displaying the target progress prompt information corresponding to the target recording progress information in the recording guide page" may be: obtaining the target progress prompt data corresponding to the target recording progress information; and drawing the target progress prompt information corresponding to the target recording progress information in the recording guide page according to the target progress prompt data to display the target progress prompt information. The target progress prompt information may include a progress bar, a video name, and a recording time.

[0160] When the target recording mode is the posture recording mode, there may be a situation where the video of the posture recording model needs to be re-recorded. Specifically, the recording guide page may also include a target re-recording control; based on this, the method of "recording a video of the target object maintaining the target posture under the posture guidance information, and obtaining a video corresponding to the posture recording mode" may be: recording a video of the target object maintaining the target posture based on the posture guidance information; in response to a click operation on the target re-recording control, stopping the step of recording a video of the target object maintaining the target posture based on the posture guidance information; returning to the step of displaying the posture guidance information until the target recording time of the posture recording mode reaches the preset time threshold, and obtaining a video corresponding to the posture recording mode.

[0161] S3. Return to the step of displaying the recording guide page corresponding to the target recording mode until each state is the target state, obtain the video in each state, and fuse the videos in each state to obtain the target video.

[0162] Among them, each state can have a corresponding recording mode, and the target recording mode can include a follow-up recording mode and a posture recording mode. Based on this, in step S3, the step of "fusing the videos in each state to obtain the object video" can be: fusing the videos corresponding to the follow-up recording mode to obtain the follow-up video corresponding to the follow-up recording mode; fusing the videos corresponding to the posture recording mode to obtain the posture video corresponding to the posture recording mode; and using the follow-up video and the posture video as the object video.

[0163] Specifically, the present application can fuse the videos corresponding to the follow-up recording mode according to the timestamps of the videos corresponding to the follow-up recording mode; similarly, the present application can fuse the videos corresponding to the gesture recording mode according to the timestamps of the videos corresponding to the gesture recording mode.

[0164] With respect to step S1 to step S3, a specific example is used for explanation here, in which the state can be a follow-up reading state and a posture state. The recording mode may include a follow-up reading recording mode and a posture recording mode. The recording mode corresponding to the follow-up reading state is a follow-up reading recording mode, and the recording mode corresponding to the posture state is a posture recording mode. Here, the example of preferentially recording the video in the follow-up reading state and then recording the video in the posture state is used for explanation. In the present application, the recording guide page may include a follow-up reading guide page and a posture guide page.

[0165] (I) For the follow-up reading guide page, this application can dynamically display the following after displaying the follow-up reading guide page: Figure 8 When the recording countdown ends, the recording countdown is hidden and the follow-up guide page is displayed. Fig. 9 The recording guide page shown is used as a follow-up reading guide page. The follow-up reading guide page can display at least one follow-up reading text corresponding to the target follow-up reading guide information, and play the follow-up reading text by voice. On this basis, the target object can follow the follow-up reading text.

[0166] During the process of the target object following the reading, the following reading guide page can also display progress prompt information. The following reading guide page can also display a re-recording control. When the target object clicks the re-recording control, the video in the following reading recording mode can be re-recorded.

[0167] (ii) For the posture guidance page, such as Fig.10 As shown, the posture guidance page can display a recording reminder message 101 for entering the posture recording mode. Then, when the time threshold is reached, the recording reminder message is hidden, and the target recording countdown information is dynamically displayed. The target dynamic countdown can be specifically referred to Figure 8; When the target recording countdown information reaches the target stop condition, the target recording countdown information is hidden and the posture guidance page is displayed. Then, in the posture guidance page, the posture guidance information can be played by voice. Under the guidance of the posture guidance information, the target object can maintain the target posture corresponding to the posture guidance information, such as the speech posture, to complete the video corresponding to the posture recording mode.

[0168] like Fig.11 As shown, during the process of the target object recording the video corresponding to the gesture recording mode, the gesture guide page can also display the target progress prompt information 111. The gesture guide page can also display the target re-recording control 112. When the target object clicks the target re-recording control 112, the video in the gesture recording mode can be re-recorded.

[0169] When the target re-recording control 112 is clicked, Fig.12 The recording interruption page shown is displayed, and the recording interruption page may include recording interruption information 121 and a target information confirmation control 122 corresponding to the recording interruption information 121; in response to a click operation on the target information confirmation control 122, the recording interruption information 121 is hidden, and the posture guidance page is displayed, and the step of displaying the posture guidance information is returned to execute until the target recording time of the posture recording mode reaches the preset time threshold, and the video corresponding to the posture recording mode is obtained to re-record the video of the posture recording mode.

[0170] When the video recording in the gesture recording mode is completed, the recording guide page such as the gesture guide page can display the recording end indication information, such as Fig.13 As shown, the instruction text information 131 corresponding to the recording end instruction information is displayed in the posture guidance page, and the instruction voice corresponding to the recording end instruction information is played. When the recording end instruction information reaches the preset condition, the video saving page is displayed. Fig.14 As shown, the video saving page can display the saving information. The saving information is as follows: Fig.14 The contents in the dashed box are shown in .

[0171] The indicative voice content may be, for example, "Video recording is complete. Please stay where you are and remain silent for 3 seconds. 321. Congratulations, you have completed the video recording. We have saved the video for you."

[0172] After generating the object video or saving the completed video, the target object can also confirm the object video. For example, a video confirmation page is displayed, which may include a target thumbnail of the object video, a video recording mode identifier of the object video corresponding to the target thumbnail, and confirmation indication information corresponding to the object video. A preview screen of the object video may be displayed in the target thumbnail.

[0173] Among them, Fig.15 As shown, the video thumbnails may include a follow-up video thumbnail 151 corresponding to the follow-up recording mode and a thumbnail 152 corresponding to the gesture recording mode.

[0174] When a sliding operation is performed on a target thumbnail, the video confirmation page can display confirmation indication information and video recording mode corresponding to another target thumbnail. For example, when the video confirmation page currently displays the corresponding follow-up video thumbnail 151, the video confirmation page can also include a follow-up recording mode identifier corresponding to the follow-up video thumbnail 151, such as "Clip 1 Follow-up Mode", and a first confirmation indication information 153 corresponding to the follow-up video. When a sliding operation is performed on the follow-up video thumbnail 151, the video confirmation page will display a posture video thumbnail 152 corresponding to the posture video, a posture recording mode identifier corresponding to the posture video thumbnail 152, such as "Clip 2 Body Recording", and a second confirmation indication information 154 corresponding to the posture video.

[0175] When a target thumbnail is clicked, the target video corresponding to the target thumbnail can be played in response to the click operation on the target thumbnail. For example, when the target object clicks the follow-up video thumbnail 151, the follow-up video is played; when the target object clicks the gesture video thumbnail 152, the gesture video is played.

[0176] The video confirmation page may further include a download control 155, a video re-recording control 156, and a video submission control 157. When the download control 155 is triggered, the target video corresponding to the target thumbnail currently displayed on the video confirmation page is downloaded; when the video re-recording control 156 is clicked, the step of displaying the video recording page is returned to re-record the target video; when the video submission control 157 is selected, the target video is submitted to the server.

[0177] When a return operation to the video confirmation page is detected, such as Fig.16 As shown, the return indication information is displayed, and the return indication information includes the video submission indication information and the return control 161 and the non-return control 162 corresponding to the video submission indication information. Fig.16 The content in the dashed box is shown in the figure.

[0178] When the return control 161 is clicked, in response to the click operation on the return control 161, the step of displaying the video recording page is returned to re-record the object video; when the non-return control 162 is clicked, in response to the trigger operation on the non-return control 162, the return indication information is hidden, and the video confirmation page remains displayed.

[0179] When the video submission control 157 in the video confirmation page is selected, the target video is submitted to the server and the submission page is displayed. Fig.17 As shown, the submission page will display the submission information and the save path information of the object video.

[0180] The present application can also display a video recording prompt page to prompt the target object. Specifically, the step of "displaying the video recording page" can be: displaying the video recording prompt page, the video recording prompt page includes recording prompt information corresponding to each recording mode and confirmation controls corresponding to the recording prompt information; in response to the target object's confirmation operation on the confirmation control, displaying the video recording page.

[0181] The recording prompt information may refer to the prompt key point information corresponding to each recording mode during the video recording process.

[0182] Among them, the present application can also play a prompt video after displaying the video recording prompt page. Specifically, the recording prompt information includes prompt text information and a prompt video thumbnail. After the step of "displaying the video recording prompt page", the computer device can respond to the target object's click operation on the prompt video thumbnail, display the prompt video corresponding to the prompt video thumbnail; and play the prompt video corresponding to the prompt video thumbnail.

[0183] Specifically, if Fig.18 As shown, the video recording prompt page includes the follow-up prompt text information 181 and the follow-up prompt video thumbnail 182 corresponding to the follow-up mode, the posture prompt text information 183 and the posture prompt video thumbnail 184 corresponding to the posture mode, and the confirmation control 185. Among them, the follow-up prompt video thumbnail 182 can include a video preview screen of the follow-up prompt video; the posture prompt video thumbnail 184 can include a video preview screen of the posture prompt video.

[0184] When the target object performs a trigger operation on the follow-up prompt video thumbnail 182, the follow-up prompt video corresponding to the follow-up prompt video thumbnail is played. When the target object performs a click operation on the gesture prompt video thumbnail 184, the gesture prompt video corresponding to the gesture prompt video thumbnail is played.

[0185] Specifically, when the target object performs a trigger operation on the follow-up prompt video thumbnail 182, the server may be requested to obtain the follow-up prompt video data corresponding to the follow-up prompt video thumbnail, so as to render the follow-up prompt video data and play the follow-up prompt video corresponding to the follow-up prompt video thumbnail. When the target object performs a click operation on the gesture prompt video thumbnail 184, the server may be requested to obtain the gesture prompt video data corresponding to the gesture prompt video thumbnail, so as to render the gesture prompt video data and play the gesture prompt video corresponding to the gesture prompt video thumbnail.

[0186] After step S202, the present application may train a preset object generation model based on the object video. Specifically, when the preset object generation model is an untrained object generation model, the method of training the preset object generation model based on the object video may be: sending the object video to a server so that the server trains the preset object generation model based on the object video to obtain a virtual object generation model corresponding to the target object.

[0187] Among them, the preset object model can be, for example, a self-attention neural network model, or a CNN neural network model, and so on. Among them, the preset object generation model can include multiple sub-models, for example, the preset object generation model can include a preset vocal area generation model and a preset face generation model. The training process of the preset object generation model in the server of this application can be specifically referred to the explanation in the next application embodiment, which will not be repeated here. Of course, when the computer device is a server, this application can train the preset object generation model in the computer device, and the specific process is similar to the training process of the preset object generation model in the next application embodiment, which will not be repeated here.

[0188] After step S202, the present application may update the preset object generation model based on the object video. Specifically, when the preset object generation model is a trained object generation model, the method of updating the preset object generation model based on the object video may be: sending the object video to a server so that the server updates the preset object generation model based on the object video to obtain a virtual object generation model corresponding to the target object.

[0189] Among them, the specific process of the server updating the preset object generation model based on the object video can be found in the next application embodiment "Training process of the preset object generation model in the server", which will not be repeated here.

[0190] S203: When the target voice of the target object is received, the target virtual object corresponding to the target object is displayed.

[0191] Among them, the target virtual object includes a virtual object generated based on the target voice, follow-up video and gesture video.

[0192] The target voice may be a voice input by the target object. For example, the virtual object generation page may include a voice recording control, and when a recording operation is performed on the voice recording control, the target voice of the target object is acquired in response to the input operation on the voice recording control.

[0193] The target voice may be a voice converted from an input text input by the target object. For example, the virtual object generation page may include a text input control, and when an input operation is performed on the text input control, the input text is obtained in response to an input operation on the voice recording control; the input text is converted into voice to obtain the target voice of the target object.

[0194] The target voice may also be a voice obtained by changing the original voice input by the target object.

[0195] The target voice may also be a voice extracted from the target video, for example, a voice extracted from a follow-up video of the target video. When the target voice is a voice extracted from the target video, the voice in the target video may be intercepted to obtain the target voice.

[0196] The present application can utilize the aforementioned virtual object generation model to generate a target video of a target virtual object, so that a computer device can display the target virtual object through a video. For example, in step S203, the method of the step of "displaying the target virtual object corresponding to the target object" can be, for example: sending the target voice of the target object to the server, so that the server uses the virtual object generation model to generate a target video of the target virtual object corresponding to the target object based on the target voice; receiving the target video returned by the server, and playing the target video to display the target virtual object.

[0197] Among them, the method of "the server uses a virtual object generation model to generate a target video of a target virtual object corresponding to a target object based on a target voice" can be specifically referred to the next application embodiment, which will not be repeated here.

[0198] The present application can display a virtual object generation page, which includes an object generation control; in response to the trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain object videos, which include follow-up videos and gesture videos; when the target voice of the target object is received, the target virtual object corresponding to the target object is displayed, and the target virtual object includes a virtual object generated based on the target voice, follow-up video and gesture video. Since the target object can record multiple videos of the target object in different states by triggering the object generation control in the virtual object generation page, the convenience of object video recording is improved, so that when the target voice is received, the target video of the target virtual object generated based on the target voice, follow-up video and gesture video can be quickly displayed to improve the display efficiency of the virtual digital person.

[0199] According to the method described in the above embodiment, the following is further described in detail with examples.

[0200] In this embodiment, the virtual object generating device is specifically integrated into a computer device, and the computer device is a server.

[0201] A virtual object display method, comprising:

[0202] An object video of a target object sent by a receiving terminal includes a follow-up video and a posture video; based on the follow-up video and the posture video, a preset object generation model is trained to obtain a virtual object generation model corresponding to the target object; when a target voice of the target object sent by the terminal is received, a target video of a target virtual object of the target object is generated by the virtual object generation model based on the target voice; the target video is sent to the terminal so that the terminal plays the target video to display the target virtual object.

[0203] like Fig.19 As shown, the specific process of the virtual object display method is shown in steps S301 to S304:

[0204] S301: Receive an object video of a target object sent by a terminal.

[0205] Among them, the object video includes follow-up video and posture video.

[0206] After receiving the object video of the target object sent by the terminal, the server may query the storage space corresponding to the target object based on the object identifier of the target object, and store the object video in the storage space.

[0207] After receiving the object video of the target object sent by the terminal, the server can train the preset object generation model to generate a virtual object generation model for generating the target video of the target virtual object.

[0208] Based on this, before training the preset object generation model in S302, the virtual object generation model corresponding to the target object can be queried in the server based on the object identifier of the target object; when the virtual object generation model does not exist, execute step S302.

[0209] When a virtual object generation model exists, the virtual object generation model can be directly used to predict the target video of the target virtual object based on the target voice, and the virtual object generation model can be used as the preset object generation model to update the preset object generation model to obtain an updated virtual object generation model; the updated virtual object generation model can be used as the virtual object generation model.

[0210] The process of “updating the preset object generation model to obtain an updated virtual object generation model” may be specifically referred to step S302 , which will not be described in detail here.

[0211] S302: Based on the follow-up video and the posture video, a preset object generation model is trained to obtain a virtual object generation model corresponding to the target object.

[0212] The preset object generation model may include multiple models, for example, the preset object generation model may include a preset sound region generation model and a preset face generation model. Based on this, in step S302, the method of "training the preset object generation model based on the follow-up video and the posture video to obtain a virtual object generation model corresponding to the target object" may be as shown in steps S31 to S33:

[0213] S31. Based on the follow-up video, a preset vocalization area generation model is trained to obtain a vocalization area generation model.

[0214] In step S31, the follow-up video includes multiple follow-up video frames and audio frames corresponding to the follow-up video frames. On this basis, the step of "training a preset sound region generation model based on the follow-up video to obtain a sound region generation model" can be: identifying the facial image of the target object in the follow-up video frame, and detecting the sound region of the target object in the facial image to obtain at least one sound region image; obtaining at least one sound key point corresponding to the sound region image, and constructing a sound region information set based on the sound key point; extracting the target audio data corresponding to the sound region in the audio frame, and training the preset sound region generation model based on the sound region information set and the target audio data to obtain the sound region generation model.

[0215] The sound producing area may be the lips of the target object or the mouth of the target object.

[0216] The step of "identifying the facial image of the target object in the follow-up video frame" may be performed by: for each follow-up video frame, extracting the facial image of the target object in the follow-up video frame using a face detection algorithm. The face detection algorithm may be a face recognition model.

[0217] Among them, the step of "detecting the vocal area of ​​the target object in the facial image and obtaining at least one vocal area image" can be: fitting the facial image with a preset model to obtain fitted facial information; extracting facial information from the fitted facial information; based on the facial information, obtaining the vocal area of ​​the target object to obtain at least one vocal area image.

[0218] The preset model may be a 3DMM model. The facial information after fitting may include facial shape, facial size, facial position, facial rotation angle, and coordinates of key points on the face. The facial information may include facial shape, facial size, and coordinates of key points on the face.

[0219] Among them, the step of "obtaining the vocal area of ​​the target object based on facial information and obtaining at least one vocal area image" can be carried out as follows: based on the facial information, normalizing the facial image to obtain a normalized facial image; identifying the vocal area of ​​the target object in the normalized facial image to obtain at least one vocal area image.

[0220] Among them, a computer vision algorithm can be used to normalize the facial image based on the facial information to obtain a normalized facial image. The present application can normalize each facial image to a normalized facial image with the front view and the same size. The normalized facial image can be as follows Fig. 20 shown.

[0221] For each normalized facial image, the present application can use a vocalization area recognition model to detect the vocalization area of ​​the target object in the normalized facial image. There are many ways to obtain the vocalization area image, for example, extracting the image area of ​​the vocalization area in the normalized facial image, and using the image area as the vocalization area image; for another example, calibrating the position information of the vocalization area in the normalized facial image, obtaining a calibrated facial image, and using the facial image after calibrating as the vocalization area image.

[0222] Among them, the vocal area recognition model can be a trained neural network model, such as a CNN neural network model.

[0223] Among them, the method of the step of "obtaining at least one vocal key point corresponding to the vocal area image" can be: for each normalized facial image, using a key point detection algorithm, to detect the vocal key points of the vocal area in the normalized facial image; and using the vocal key points of the vocal area as the vocal key points corresponding to the vocal area image.

[0224] The step of "building a vocalization area information set based on vocalization key points" may be performed by binding the vocalization area image and the vocalization key points to obtain the vocalization area information set. Alternatively, the vocalization key points are used as the vocalization area information set.

[0225] Among them, the step of "extracting target audio data corresponding to the vocalization area in the audio frame" can be: determining the target follow-up video frame corresponding to the vocalization area; screening out the target audio frame corresponding to the target follow-up video frame in the audio frame; extracting the target audio data corresponding to the vocalization area in the target audio frame.

[0226] Among them, the step of "training the preset sound area generation model based on the sound area information set and the target audio data to obtain the sound area generation model" can be as follows: using the preset sound area generation model to predict according to the target audio data to obtain the predicted sound key points; extracting the target sound key points corresponding to the target audio data from the sound area information set; calculating the target loss between the target sound key points and the predicted sound key points; according to the target loss, training the preset sound area generation model to obtain the sound area generation model.

[0227] Among them, a loss function can be used to calculate the target loss between the target utterance key point and the predicted utterance key point.

[0228] S32. According to the posture video, a preset facial generation model is trained to obtain a facial generation model.

[0229] Among them, the posture video includes multiple frames of posture video frames. Based on this, in step S32, the method of "training the preset facial generation model according to the posture video to obtain the facial generation model" can be: identifying the facial information of the target object in the posture video frame; extracting the target facial image of the target object in the posture video frame according to the facial information; training the preset facial generation model according to the target facial image and preset vocalization key points to obtain the facial generation model.

[0230] The step of "identifying facial information of the target object in the posture video frame" can be: identifying the facial image of the target object in the posture video frame; fitting the facial image using a preset model to obtain fitted facial information, and using the fitted facial information as the facial information.

[0231] Among them, a face recognition algorithm can be used to recognize the face image of the target object in the posture video frame.

[0232] The fitted face information may include face shape, face size and coordinates of vertices on the face.

[0233] After the facial information of the target object is identified in the pose video frame, the target facial image of the target object can be extracted. Specifically, the step of "extracting the target facial image of the target object in the pose video frame based on the facial information" can be: determining the vertex coordinates of the target vertex based on the facial information; and extracting the target facial image of the target object in the pose video frame based on the vertex coordinates.

[0234] Among them, the regional coordinates of the sound emitting area of ​​the target object can be identified according to the face information; the target vertex corresponding to the regional coordinates can be determined according to the regional coordinates; and the vertex coordinates of the target vertex can be extracted.

[0235] Among them, according to the vertex coordinates, lines can be connected in the posture video frame to obtain a connection area; the connection area is set to black to obtain a target facial image of the target object.

[0236] Among them, the method of "training the preset facial generation model according to the target facial image and the preset vocal key points to obtain the facial generation model" can be: obtaining a label image corresponding to the target facial image; fusing the target facial image and the preset vocal key points to obtain a fused image; training the preset facial generation model according to the label image and the fused image to obtain the facial generation model.

[0237] Among them, the label image includes the face of the target object, and the key points corresponding to the vocalization area of ​​the target object in the label image can be the same as the preset vocalization key points. It can be understood here that the present application can use the vocalization key points in the aforementioned vocalization area information set as preset vocalization key points, and the object face image corresponding to the vocalization key points as the label image. It can be understood here that the object face image can include the face image extracted and processed in the follow-up video frame corresponding to the vocalization key points.

[0238] Among them, the step of "training a preset facial generation model according to the label image and the fused image to obtain a facial generation model" can be: using the preset facial generation model to predict facial correction based on the fused image to obtain a predicted corrected facial image; calculating the candidate loss between the predicted corrected facial image and the label image; according to the candidate loss, converging the preset facial generation model to obtain the facial generation model.

[0239] S33, using the vocalization area generation model and the facial generation model as virtual object generation models corresponding to the target object.

[0240] S303: When a target voice of a target object sent by the terminal is received, a target video of a target virtual object of the target object is generated by using a virtual object generation model based on the target voice.

[0241] When the target voice sent by the terminal is received, the server can generate a target video of the target virtual object based on the target voice. Specifically, in step S303, the step of "generating a target video of the target virtual object of the target object based on the target voice using a virtual object generation model" can be: based on the target voice, using a sound area generation model to predict the target sound area image of the target object; obtaining the original virtual object video corresponding to the target object, and identifying the target facial information of the original virtual object in the original virtual object video; based on the target facial information and the target sound area image, using a facial generation model to perform facial correction on the target object to obtain a corrected facial image, and based on the corrected facial image, constructing the target video of the target virtual object of the target object.

[0242] The original virtual object video includes a plurality of original virtual object video frames. The target facial information may include the object face size, the object face rotation angle, and the three-dimensional vertex coordinates of the object face of the original virtual object. The target facial information may be information carried by the original virtual object, or may be information obtained by fitting the original virtual object video frame with the 3DMM model.

[0243] Among them, there are many ways to perform the step of "based on the target facial information and the target vocal area image, using the facial generation model to perform facial correction on the target object to obtain a corrected facial image", for example, it can be: based on the target facial information, determine the candidate vertices of the original virtual object; according to the vertex coordinates of the candidate vertices and the original virtual object video frame, extract the virtual object facial image; fuse the virtual object facial image and the target vocal area image to obtain a fused facial image; based on the fused facial image, use the facial generation model to perform facial correction on the target object to obtain a corrected facial image.

[0244] Alternatively, a face information set corresponding to the target object is obtained, the face information set including a target facial image of the target object and facial information corresponding to the target facial image; in the face information set, target face information corresponding to the target facial information is determined; in the face information set, a target facial image corresponding to the target facial information is extracted, and the target facial image corresponding to the target facial information is used as the object facial image; the object facial image and the target vocal area image are fused to obtain a fused object facial image; and a facial generation model is used to perform facial correction on the target object to obtain a corrected facial image.

[0245] Among them, the step of "based on the target speech, using the sound area generation model to predict the target sound area image of the target object" can be as follows: obtaining a sound area information set, the sound area information set including sound key points and sound area images corresponding to the sound key points; based on the target speech, using the sound area generation model to predict the predicted key points corresponding to the sound area of ​​the target object; extracting candidate sound area images corresponding to the predicted key points from the sound area information set; and determining the target sound area image of the target object based on the candidate sound area images.

[0246] The step of "extracting candidate vocal region images corresponding to predicted key points from the vocal region information set" may be performed by: calculating the similarity between vocal key points and predicted key points in the vocal region information set; when the similarity is greater than a preset similarity threshold, taking the vocal key points as candidate key points; extracting vocal region images corresponding to the candidate key points from the vocal region information set, and taking the vocal region images corresponding to the candidate key points as candidate vocal region images. Similarity functions such as cosine similarity, Gaussian kernel similarity, Euclidean distance function, etc. may be used to calculate the similarity between vocal key points and predicted key points in the vocal region information set.

[0247] Among them, the candidate sound area image may include multiple candidate sound area images; the step of "determining the target sound area image of the target object based on the candidate sound area image" can be done by: according to the pixel values ​​of the pixel points in the candidate sound area image, fusing the multiple candidate sound area images to obtain the fused sound area image, and using the fused sound area image as the target sound area image of the target object.

[0248] Among them, the step of "constructing a target video of a target virtual object of the target object based on the corrected facial image" can be performed by: fusing the corrected facial image with the original virtual object video frame to obtain a fused virtual object video frame; and generating a target video of the target virtual object of the target object based on the fused virtual object video frame.

[0249] S304: Send the target video to the terminal so that the terminal plays the target video to display the target virtual object.

[0250] In step S304, the target video is sent to the terminal, and the terminal renders the target video to play the target video to display the target virtual object.

[0251] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0252] The embodiment of the present application can receive an object video of a target object sent by a terminal, and the object video includes a follow-up video and a posture video; based on the follow-up video and the posture video, a preset object generation model is trained to obtain a virtual object generation model corresponding to the target object; when a target voice of the target object sent by the terminal is received, a target video of a target virtual object of the target object is generated by the virtual object generation model based on the target voice; the target video is sent to the terminal so that the terminal plays the target video to display the target virtual object. Since the example of the present application can train the preset object generation model using the follow-up video and the posture video sent by the terminal, when the target voice of the target object sent by the terminal is received, the trained virtual object generation model can be used to generate a target video of the target virtual object of the target object based on the target voice, so that the terminal can quickly display the target video of the target virtual object to improve the display efficiency of the virtual digital person.

[0253] According to the method described in the above embodiment, the following examples are given to further explain in detail. Fig.21 As shown, a method for generating a virtual object, as shown in steps S401 to S410:

[0254] S401: The terminal displays a virtual object generation page.

[0255] The virtual object generation page includes an object generation control.

[0256] In step S401, there are multiple ways to "display the virtual object generation page". For example, an embodiment of the present application can display a virtual object generation operation page, which may include operation guide information for generating the virtual object and a target control corresponding to the operation guide information; in response to a selection operation of the target object on the target control, the virtual object generation page is displayed.

[0257] In an embodiment of the present application, before generating a target virtual object, object information of the target virtual object to be generated may also be entered. For example, the virtual object generation page may also include an object information entry control; in response to an input operation on the object information entry control, object information of the target virtual object to be generated is generated and the object information is displayed.

[0258] In an embodiment of the present application, before generating a target virtual object, a shooting mode for the target object can also be selected. For example, the virtual object generation page can also include a mode selection control for the shooting mode; in response to the target object's selection operation on the mode selection control, the shooting mode of the target object is determined.

[0259] S402: In response to a triggering operation of the target object on the object generation control, the terminal records multiple videos of the target object in different states to obtain an object video.

[0260] For step S402, specifically, the terminal displays a video recording page in response to the trigger operation of the target object on the object generation control, and the video recording page includes recording instruction information and recording controls; in response to the recording operation of the target object on the recording control, the video in each state corresponding to the recording instruction information is recorded to obtain the object video.

[0261] The present application may select a method for displaying the recording instruction information after the step of "displaying the video recording page". For example, when the video recording page includes target text corresponding to the recording instruction information, in response to a trigger operation on a text display control, the target text is hidden, and the recording instruction information is played by voice; when the video recording page does not include the target text corresponding to the recording instruction information, in response to a trigger operation on a text display control, the target text is displayed in the video recording page.

[0262] After the video recording page is displayed, when the target object performs a recording operation on the recording control in the video recording page, the application can record the video in each state corresponding to the recording instruction information to obtain the target video. Specifically, the recording guide page corresponding to the target recording mode is displayed; the video of the target object in the target recording mode is recorded to obtain the video corresponding to the target recording mode; the step of returning to displaying the recording guide page corresponding to the target recording mode is returned until each state is the target state, the video in each state is obtained, and the videos in each state are merged to obtain the target video.

[0263] After displaying the recording guide page corresponding to the target recording mode, the present application can dynamically display the recording countdown information for the target object. For example, the recording countdown information can be a countdown of 3 seconds. Accordingly, when the recording countdown information triggers the stop condition, the recording guide page corresponding to the target recording mode is displayed.

[0264] After the application displays the recording guide page corresponding to the target recording mode, the video of the target object in the target recording mode can be recorded to obtain the video corresponding to the target recording mode. Specifically, when the target recording mode is the follow-up recording mode, the target follow-up guidance information is displayed, and the video of the target object following the target follow-up guidance information is recorded to obtain the video corresponding to the follow-up recording mode. When the target recording mode is the posture recording mode, the posture guidance information is displayed, and the video of the target object maintaining the target posture under the posture guidance information is recorded to obtain the video corresponding to the posture recording mode.

[0265] In the present application, the object video may include a follow-up video and a posture video. Specifically, the present application may fuse the videos corresponding to the follow-up recording mode to obtain the follow-up video corresponding to the follow-up recording mode; fuse the videos corresponding to the posture recording mode to obtain the posture video corresponding to the posture recording mode; and use the follow-up video and the posture video as the object video.

[0266] For step S402, an actual example can be used here to illustrate, taking the target object as a user as an example. For example, when the user clicks the recording control, a countdown of 5 seconds will be displayed on the terminal to facilitate the user to position himself, and then the terminal starts voice playback and follow-up guidance information to record the follow-up video.

[0267] When the recording of the follow-up video is completed, a voice prompt can be given to the user, such as "Please keep your position unchanged. The body recording phase will automatically start after the countdown. Please keep speaking for 1 minute. The specific voice part can be arbitrary, but you need to keep your eyes looking directly at the camera and your body in a natural state." Then, after the countdown of 3 seconds, the posture video will be recorded. The user can continue to record the posture video in the original position. After both the follow-up video and the posture video are finished, no user operation is required, and they will automatically end and be saved.

[0268] S403: The terminal sends the object video to the server.

[0269] S404: The server receives the object video of the target object sent by the terminal.

[0270] S405: The server trains a preset object generation model based on the follow-up video and the posture video included in the object video to obtain a virtual object generation model corresponding to the target object.

[0271] For step S405, the server can train the preset vocal area generation model based on the follow-up video to obtain the vocal area generation model; train the preset facial generation model according to the posture video to obtain the facial generation model; and use the vocal area generation model and the facial generation model as the virtual object generation model corresponding to the target object.

[0272] For the sending area generation model, the present application can identify the facial image of the target object in the follow-up video frame, and detect the vocalization area of ​​the target object in the facial image to obtain at least one vocalization area image; obtain at least one vocalization key point corresponding to the vocalization area image, and construct a vocalization area information set based on the vocalization key point; extract the target audio data corresponding to the vocalization area in the audio frame, and train the preset vocalization area generation model based on the vocalization area information set and the target audio data to obtain the vocalization area generation model.

[0273] Among them, the vocalization area image and the vocalization key points can be bound to obtain a vocalization area information set.

[0274] For the facial generation model, the present application can identify the facial information of the target object in the posture video frame; based on the facial information, extract the target facial image of the target object in the posture video frame; based on the target facial image and preset vocalization key points, train the preset facial generation model to obtain the facial generation model.

[0275] The above-mentioned vocal region generation model and facial generation model are described with specific examples. Fig. 22 shown.

[0276] (1) For the vocalization area generation model, first, an audio-visual synchronization detection algorithm is used to detect whether the audio in the follow-up video is synchronized with the lip shape of the target object, wherein the audio-visual synchronization detection algorithm can be an audio-visual synchronization detection model, and the audio-visual synchronization detection model can be a Sync-net neural network model. When the audio in the follow-up video is not synchronized with the lip shape of the target object, silence is added to the audio or audio frames such as the audio frame at the beginning of the sentence are deleted to achieve lip synchronization between the audio and the target object.

[0277] When the audio in the follow-up video is synchronized with the lip shape of the target object, a face detection algorithm is used to detect the position of the target object's face in the follow-up video frame, and then the face image of the target object is extracted from the follow-up video frame. Then, the facial image is fitted using a 3DMM model to obtain fitted facial information; facial information is extracted from the fitted facial information. Then, based on the facial information, the facial image is normalized to obtain a normalized facial image; the vocalization area of ​​the target object is identified in the normalized facial image to obtain at least one vocalization area image; in the normalized facial image, the vocalization key points of the vocalization area are detected; the vocalization key points of the vocalization area are used as the vocalization key points corresponding to the vocalization area image. Then, the vocalization key points and the vocalization area image are bound to construct a vocalization area information set.

[0278] Then, the target audio data corresponding to the vocalization area is extracted from the audio frame, and based on the vocalization area information set and the target audio data, the preset vocalization area generation model is trained to obtain the vocalization area generation model.

[0279] (2) For the facial generation model, first, a face recognition algorithm can be used to identify the face image of the target object in the pose video frame; then, the 3DMM model is used to fit the face image to obtain the fitted face information, and the fitted face information is used as the face information; according to the face information, the vertex coordinates of the three-dimensional target vertices are determined; according to the vertex coordinates, lines are connected in the pose video frame to obtain the line area; the line area is set black to obtain the target face image of the target object. Among them, the target face image is as follows: Fig.23 shown.

[0280] Then, obtain the label image corresponding to the target facial image; fuse the target facial image and the preset vocalization key points to obtain a fused image; train the preset facial generation model based on the label image and the fused image to obtain a facial generation model. Fig.24 shown.

[0281] S406: The terminal obtains the target voice of the target object and sends the target voice to the server.

[0282] With respect to step S406, the terminal may display an audio input page, which includes an audio input control; and the terminal generates a target voice of the target object in response to an input operation of the target object on the audio input control.

[0283] S407: The server receives the target voice of the target object.

[0284] S408: The server generates a target video of a target virtual object of the target object by using a virtual object generation model based on the target voice.

[0285] Step S303 is described below with a specific example. Fig.25 As shown, when the target voice of the target object sent by the terminal is received, the original virtual object video with the same length as the target voice playback time is obtained. The original virtual object video can be obtained by intercepting the original virtual object video clip, or continuously playing it forward or backward to extend it.

[0286] Then, based on the target speech, the prediction key points corresponding to the vocal area of ​​the target object are predicted using the vocal area generation model; based on the prediction key points, N candidate key points and candidate vocal area images corresponding to each candidate key point are retrieved from the vocal area information set. Based on the similarity between the candidate key points and the prediction key points, the weight value of the candidate vocal area image corresponding to the candidate key point is determined; then, based on the weight value, the pixel values ​​of the pixels of the N candidate vocal area images are weighted averaged to obtain a fused vocal area image, and the fused vocal area image is used as the target vocal area image of the target object. Among them, the target vocal area image is the mouth image of the target object, and the mouth image includes the lip shape and teeth of the target object.

[0287] Then, the target facial information of the original virtual object is identified in the original virtual object video. Next, a facial information set corresponding to the target object is obtained, the facial information set includes a target facial image of the target object and facial information corresponding to the target facial image; in the facial information set, the target facial information corresponding to the target facial information is determined; in the facial information set, the target facial image corresponding to the target facial information is extracted, and the target facial image corresponding to the target facial information is used as the object facial image; the object facial image and the target vocalization area image are fused to obtain a fused object facial image; and the facial generation model is used to perform facial correction on the target object to obtain a corrected facial image.

[0288] Then, the corrected facial image is fused with the original virtual object video frame to obtain a fused virtual object video frame; and a target video of a target virtual object of the target object is generated according to the fused virtual object video frame.

[0289] S409: The server sends the target video to the terminal.

[0290] S410: The terminal receives a target video, and plays the target video to display a target virtual object.

[0291] In the embodiment of the present application, for the object video, the terminal can use the technical method of embedding HTML5 in an application such as a mini-program to use a high-definition camera to achieve segmented video recording and obtain the object video. The specific steps of the terminal to achieve segmented video recording can be described as follows:

[0292] (I) The terminal embeds an HTML5 page in the application and creates a video element tag for displaying the video. The HTML5 page may be a video recording page.

[0293] (ii) The terminal calls a camera start method in the HTML5 page to start the high-definition camera of the terminal, so as to obtain a high-definition video stream corresponding to the preview image of the target object.

[0294] The camera start method may be the navigator.mediaDevices.getUserMedia method.

[0295] (3) The terminal displays the acquired high-definition video stream on the video recording page through the video element tag for the target object to watch.

[0296] (iv) The terminal loads the high-definition audio and video stream into the video recording tool and prepares for the recording operation.

[0297] (V) When the target object clicks the recording control in the video recording page, the terminal dynamically displays the countdown information, such as a countdown of 3 seconds, and then actually executes the recording start action. This ensures that the target object has enough preparation time.

[0298] (VI) The terminal starts recording the video in the follow-up reading mode: the follow-up reading guidance information is played by using a voice playback tool, and the playback start and end events of each follow-up reading text of the follow-up reading guidance information are detected.

[0299] When the follow-up text starts playing, the terminal stops recording, waits for the follow-up text to finish playing, and then starts recording the target object following the follow-up text. When the target object finishes following, the terminal stops recording, continues playing the next follow-up text video, waits for the next follow-up text to finish playing, and then starts recording the target object following the next follow-up text video, and so on.

[0300] This application will strictly stagger the playback and recording operations of the follow-up text to avoid them happening at the same time to ensure the correct execution order.

[0301] (VII) After the terminal finishes recording the video in the follow-up reading mode, it starts recording the video in the gesture mode.

[0302] In this application, the local cache area used to cache object videos will be cleared regularly to prevent the video file being recorded from being too large and exceeding the browser storage limit.

[0303] Based on the above, this application has many advantages, as follows:

[0304] First, the present application can use segmented video recording to record multiple videos of the target object in different states, which can improve the convenience of recording while allowing users to experience easy and efficient shooting. Moreover, the object video captured by the present application can train a virtual object generation model corresponding to the target object, which can maximize the production effect of the virtual digital human in the later stage.

[0305] Specifically, on the one hand, segmented video recording splits the video recording that originally needs to be shot in one go into two or more segments for separate recording, decouples the follow-up video and posture video, and records in segments, which greatly reduces the difficulty of user expressive control during recording; for example, first we will segment the digital human image problem that is most difficult for users to record into a video recording of up to 1 minute, first reducing user troubles in terms of duration; secondly, in this 1 minute of posture recording time, we have basically no requirements for the user's speech part, and the user can even repeat a sentence or count to record, so that the user can focus more on the body shape and natural eye level, thereby ensuring that the virtual digital human has a better display effect in the later stage.

[0306] On the other hand, voice dialogue is used to guide users in lip reading, shooting movements and precautions, etc., to solve the problem of users shooting without a script and not knowing how to record specifically, which leads to overall unnaturalness. For example, during lip shooting, users only need to repeat the voice content, without ensuring the body shape and eye expression, or even the reading content does not need to be completely correct. It is extremely friendly for novices and white users who are trying digital human shooting for the first time, and it is conducive to users to complete the video shooting before digital human production more conveniently, quickly, easily and efficiently.

[0307] Secondly, in the existing related technologies, the video samples of the object that is captured do not meet the standards and cannot be used to produce virtual digital people account for more than 30%, which will affect the user's enthusiasm for recording. However, this application can effectively improve the user's enthusiasm for use, and the efficiency of the object video captured by this application can reach more than 95%.

[0308] Thirdly, this application can solve the problem that WeChat applet cannot call high-definition camera by using the technical solution of embedding HTML5 page in applet. In addition, this application can break through the limitation of browser storage size by regularly clearing the local cache area used to cache object videos.

[0309] In summary, the present application can display a virtual object generation page, which includes an object generation control; in response to the trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain an object video, which includes a follow-up video and a posture video; when the target voice of the target object is received, the target virtual object corresponding to the target object is displayed, and the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the posture video. Since the target object can record multiple videos of the target object in different states by triggering the object generation control in the virtual object generation page, the convenience of recording the object video is improved, so that when the target voice is received, the target video of the target virtual object generated based on the target voice, the follow-up video and the posture video can be quickly displayed to improve the display efficiency of the virtual digital person.

[0310] In order to better implement the above method, an embodiment of the present application also provides a virtual object generation device, which can be integrated in a computer device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer.

[0311] For example, Fig.26 As shown, the virtual object generating device may include a display unit 301, a recording unit 302 and a first display unit 303, as follows:

[0312] (1) Display unit 301;

[0313] The display unit 301 may be used to display a virtual object generation page, where the virtual object generation page includes an object generation control.

[0314] In some embodiments, the display unit 301 can be used to display a video recording prompt page, which includes recording prompt information corresponding to each recording mode and confirmation controls corresponding to the recording prompt information; in response to the target object's confirmation operation on the confirmation control, the video recording page is displayed.

[0315] In some embodiments, the recording prompt information includes prompt text information and a prompt video thumbnail; the display unit 301 can be used to respond to the target object's click operation on the prompt video thumbnail, display the prompt video corresponding to the prompt video thumbnail; and play the prompt video corresponding to the prompt video thumbnail.

[0316] (2) recording unit 302;

[0317] The recording unit 302 can be used to record multiple videos of the target object in different states in response to the triggering operation of the target object on the object generation control to obtain the object video, which includes the follow-up video and the posture video.

[0318] In some embodiments, the recording unit 302 can be used to display a video recording page, which includes recording indication information and recording controls; in response to the target object's recording operation on the recording control, the video in each state corresponding to the recording indication information is recorded to obtain the object video.

[0319] In some embodiments, the video recording page includes a text display control; the recording unit 302 can also be used to hide the target text and play the recording instruction information by voice in response to a trigger operation on the text display control when the video recording page includes the target text corresponding to the recording instruction information; when the video recording page does not include the target text corresponding to the recording instruction information, the target text is displayed in the video recording page in response to a trigger operation on the text display control.

[0320] In some embodiments, the recording indication information indicates the recording order of each state; the recording unit 302 can be used to display a recording guide page corresponding to a target recording mode, the target recording mode including a recording mode corresponding to a target state determined in the state based on the recording order; record a video of the target object in the target recording mode to obtain a video corresponding to the target recording mode; return to the step of displaying the recording guide page corresponding to the target recording mode until each state is the target state, obtain the video in each state, and fuse the videos in each state to obtain the object video.

[0321] In some embodiments, the recording unit 302 can be used to filter out a target state from the states based on a recording order; determine a target recording mode corresponding to the target object according to the target state; and display a recording guide page corresponding to the target recording mode.

[0322] In some embodiments, the recording unit 302 may be used to dynamically display recording countdown information for a target object; when the recording countdown information triggers a stop condition, a recording guide page corresponding to the target recording mode is displayed.

[0323] In some embodiments, the recording unit 302 can be used to display target follow-up guidance information and record a video of the target object following the target follow-up guidance information when the target recording mode is a follow-up recording mode, so as to obtain a video corresponding to the follow-up recording mode; when the target recording mode is a posture recording mode, it can display posture guidance information and record a video of the target object maintaining the target posture under the posture guidance information to obtain a video corresponding to the posture recording mode.

[0324] In some embodiments, the recording unit 302 can be used to obtain target follow-up guidance information corresponding to the target object; when the video recording page includes the target text corresponding to the recording instruction information, at least one follow-up text corresponding to the target follow-up guidance information is displayed in the recording guidance page, or the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played; when the video recording page does not include the target text corresponding to the recording instruction information, the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played in the recording guidance page.

[0325] In some embodiments, the recording unit 302 may be used to obtain recording progress information of the target object in the follow-up reading mode; and display progress prompt information corresponding to the recording progress information in the recording guide page.

[0326] In some embodiments, the recording guidance page also includes a re-recording control; the recording unit 302 can be used to record a video of the target object following the reading based on the target follow-up guidance information; in response to a trigger operation on the re-recording control, stop executing the step of recording the video of the target object following the reading based on the target follow-up guidance information; return to execute the step of displaying the target follow-up guidance information until the target follow-up guidance information is fully displayed, and obtain the video corresponding to the follow-up recording mode.

[0327] (3) a first display unit 303;

[0328] The first display unit 303 can be used to display a target virtual object corresponding to the target object when a target voice of the target object is received. The target virtual object includes a virtual object generated based on the target voice, the follow-up video and the gesture video.

[0329] In some embodiments, the first display unit 303 may be used to send the object video to the server so that the server can train a preset object generation model based on the object video to obtain a virtual object generation model corresponding to the target object.

[0330] In some embodiments, the first display unit 303 can be used to send the target voice of the target object to the server, so that the server uses the virtual object generation model to generate a target video of the target virtual object corresponding to the target object based on the target voice; receive the target video returned by the server, and play the target video to display the target virtual object.

[0331] As can be seen from the above, the display unit 301 of the present application can be used to display a virtual object generation page, and the virtual object generation page includes an object generation control; the recording unit 302 can be used to respond to the trigger operation of the target object on the object generation control, record multiple videos of the target object in different states, and obtain the object video, and the object video includes a follow-up video and a posture video; the first display unit 303 can be used to display the target virtual object corresponding to the target object when the target voice of the target object is received, and the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the posture video. Since the target object can record multiple videos of the target object in different states by triggering the object generation control in the virtual object generation page, the convenience of recording the object video is improved, so that when the target voice is received, the target video of the target virtual object generated based on the target voice, the follow-up video and the posture video can be quickly displayed to improve the display efficiency of the virtual digital person.

[0332] In order to better implement the above method, an embodiment of the present application further provides a virtual object generation device, which can be integrated in a computer device, such as a server.

[0333] For example, Fig. 27 As shown, the virtual object generation device may include a receiving unit 501, a training unit 502, a generation unit 503 and a second display unit 504, as follows:

[0334] (1) receiving unit 501;

[0335] The receiving unit 501 may be configured to receive an object video of a target object sent by a terminal, where the object video includes a follow-up video and a gesture video.

[0336] (2) training unit 502;

[0337] The training unit 502 can be used to train the preset object generation model based on the follow-up video and the posture video to obtain a virtual object generation model corresponding to the target object.

[0338] In some embodiments, the preset object generation model includes a preset vocal area generation model and a preset facial generation model; the training unit 502 can be used to train the preset vocal area generation model based on the follow-up video to obtain the vocal area generation model; train the preset facial generation model according to the posture video to obtain the facial generation model; and use the vocal area generation model and the facial generation model as the virtual object generation model corresponding to the target object.

[0339] In some embodiments, the follow-up video includes multiple follow-up video frames and audio frames corresponding to the follow-up video frames; the training unit 502 can be used to identify the facial image of the target object in the follow-up video frame, and detect the vocalization area of ​​the target object in the facial image to obtain at least one vocalization area image; obtain at least one vocalization key point corresponding to the vocalization area image, and construct a vocalization area information set based on the vocalization key point; extract the target audio data corresponding to the vocalization area in the audio frame, and train the preset vocalization area generation model based on the vocalization area information set and the target audio data to obtain the vocalization area generation model.

[0340] (3) generating unit 503;

[0341] The generating unit 503 may be configured to generate a target video of a target virtual object of the target object by using a virtual object generating model based on the target voice when receiving the target voice of the target object sent by the terminal;

[0342] In some embodiments, the generation unit 503 can be used to predict the target vocal area image of the target object based on the target speech by using the vocal area generation model; obtain the original virtual object video corresponding to the target object, and identify the target facial information of the original virtual object in the original virtual object video; based on the target facial information and the target vocal area image, use the facial generation model to perform facial correction on the target object to obtain the corrected facial image, and based on the corrected facial image, construct the target video of the target virtual object of the target object.

[0343] (4) a second display unit 504;

[0344] The second display unit 504 may be used to send the target video to the terminal so that the terminal plays the target video to display the target virtual object.

[0345] The receiving unit 501 of the embodiment of the present application can be used to receive the object video of the target object sent by the terminal, and the object video includes a follow-up video and a posture video; the training unit 502 can be used to train the preset object generation model based on the follow-up video and the posture video to obtain the virtual object generation model corresponding to the target object; the generation unit 503 can be used to generate the target video of the target virtual object of the target object based on the target voice when the target voice of the target object sent by the terminal is received; the second display unit 504 can be used to send the target video to the terminal so that the terminal plays the target video to display the target virtual object. Since the example of the present application can use the follow-up video and the posture video sent by the terminal to train the preset object generation model, in this way, when the target voice of the target object sent by the terminal is received, the trained virtual object generation model can be used to generate the target video of the target virtual object of the target object based on the target voice, so that the terminal can quickly display the target video of the target virtual object to improve the display efficiency of the virtual digital person.

[0346] The present application also provides an electronic device, such as Fig.28 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0347] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Fig.28 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0348] The processor 401 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the electronic device and processes data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0349] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0350] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0351] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0352] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, as follows:

[0353] A virtual object generation page is displayed, the virtual object generation page includes an object generation control; in response to a trigger operation of the target object on the object generation control, multiple videos of the target object in different states are recorded to obtain an object video, the object video includes a follow-up video and a posture video; when a target voice of the target object is received, a target virtual object corresponding to the target object is displayed, the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the posture video.

[0354] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0355] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0356] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any virtual object display method provided in the embodiment of the present application.

[0357] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0358] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0359] Since the instructions stored in the computer-readable storage medium can execute the steps in any virtual object display method provided in the embodiments of the present application, the beneficial effects that can be achieved by any virtual object display method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0360] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0361] The above is a detailed introduction to a virtual object display method and related equipment provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A virtual object display method, characterized in that: include: Displaying a virtual object generation page, wherein the virtual object generation page includes an object generation control; In response to a trigger operation of a target object on the object generation control, a plurality of videos of the target object in different states are recorded to obtain an object video, wherein the object video includes a follow-up video and a posture video; When the target voice of the target object is received, a target virtual object corresponding to the target object is displayed, wherein the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the gesture video.

2. The virtual object display method according to claim 1, characterized in that: The step of recording multiple videos of the target object in different states to obtain the object video includes: Displaying a video recording page, wherein the video recording page includes recording instruction information and recording controls; In response to the target object's recording operation on the recording control, the video in each state corresponding to the recording instruction information is recorded to obtain the object video.

3. The virtual object display method according to claim 2, characterized in that: The video recording page includes a text display control; after displaying the video recording page, the method further includes: When the video recording page includes the target text corresponding to the recording instruction information, in response to a trigger operation on the text display control, the target text is hidden, and the recording instruction information is played by voice; When the video recording page does not include the target text corresponding to the recording indication information, the target text is displayed in the video recording page in response to a triggering operation on the text display control.

4. The virtual object display method according to claim 2, characterized in that: The recording indication information indicates the recording order of each state; The recording of the video in each state corresponding to the recording indication information to obtain the object video includes: displaying a recording guide page corresponding to a target recording mode, wherein the target recording mode includes a recording mode corresponding to a target state determined in the state based on the recording sequence; Record a video of the target object in the target recording mode to obtain a video corresponding to the target recording mode; Return to the step of displaying the recording guide page corresponding to the target recording mode until each state is the target state, obtain the video in each state, and merge the videos in each state to obtain the object video.

5. The virtual object display method according to claim 4, characterized in that: The recording guide page corresponding to the display target recording mode includes: Based on the recording sequence, filtering out a target state from the states; Determining a target recording mode corresponding to the target object according to the target state; Display the recording guide page corresponding to the target recording mode.

6. The virtual object display method according to claim 4, characterized in that: After displaying the recording guide page corresponding to the target recording mode, the method further includes: Dynamically displaying the recording countdown information for the target object; The displaying of the recording guide page corresponding to the target recording mode includes: when the recording countdown information triggers a stop condition, displaying the recording guide page corresponding to the target recording mode.

7. The virtual object display method according to claim 4, characterized in that: The recording of the video of the target object in the target recording mode to obtain the video corresponding to the target recording mode includes: When the target recording mode is a follow-up recording mode, display target follow-up guidance information, and record a video of the target object following the target follow-up guidance information to obtain a video corresponding to the follow-up recording mode; When the target recording mode is a posture recording mode, posture guidance information is displayed, and a video of the target object maintaining a target posture under the posture guidance information is recorded to obtain a video corresponding to the posture recording mode.

8. The virtual object display method according to claim 7, characterized in that: The display target follow-up guidance information includes: Obtaining target follow-up guidance information corresponding to the target object; When the video recording page includes the target text corresponding to the recording instruction information, displaying at least one follow-up text corresponding to the target follow-up guidance information in the recording guidance page, or playing the text voice of at least one follow-up text corresponding to the target follow-up guidance information; When the video recording page does not include the target text corresponding to the recording instruction information, the text voice of at least one follow-up text corresponding to the target follow-up guidance information is played in the recording guide page.

9. The virtual object display method according to claim 7, characterized in that: After the display target follows the guidance information, the method further includes: Obtaining recording progress information of the target object in the follow-up reading mode; The progress prompt information corresponding to the recording progress information is displayed in the recording guide page.

10. The virtual object display method according to claim 7, characterized in that: The recording guide page also includes a re-recording control; The step of recording a video of the target object following the target reading guidance information to obtain a video corresponding to the following reading recording mode includes: Recording a video of the target subject following the reading based on the target following-reading guidance information; In response to a trigger operation on the re-recording control, stopping the step of recording the video of the target object following the reading based on the target following-reading guidance information; Return to the step of displaying the target follow-up guidance information until all the target follow-up guidance information is displayed, and obtain the video corresponding to the follow-up recording mode.

11. The virtual object display method according to claim 2, characterized in that: The video recording page is displayed, including: Displaying a video recording prompt page, wherein the video recording prompt page includes recording prompt information corresponding to each recording mode and a confirmation control corresponding to the recording prompt information; In response to the target object's confirmation operation on the confirmation control, the video recording page is displayed.

12. The virtual object display method according to claim 11, characterized in that: The recording prompt information includes prompt text information and prompt video thumbnail; After displaying the video recording prompt page, the method further includes: In response to a click operation of the target object on the prompt video thumbnail, displaying a prompt video corresponding to the prompt video thumbnail; Play the prompt video corresponding to the prompt video thumbnail.

13. The virtual object display method according to claim 1, characterized in that: After recording multiple videos of the target object in different states to obtain the object video, the method further includes: The object video is sent to a server so that the server can train a preset object generation model based on the object video to obtain a virtual object generation model corresponding to the target object.

14. The virtual object display method according to claim 13, characterized in that: The displaying of the target virtual object corresponding to the target object includes: Sending the target voice of the target object to the server, so that the server uses the virtual object generation model to generate a target video of the target virtual object corresponding to the target object based on the target voice; The target video returned by the server is received, and the target video is played to display the target virtual object.

15. A method for displaying a virtual object, characterized in that: include: An object video of a target object sent by a receiving terminal, wherein the object video includes a follow-up video and a posture video; Based on the reading video and the posture video, a preset object generation model is trained to obtain a virtual object generation model corresponding to the target object; When receiving the target voice of the target object sent by the terminal, generating a target video of the target virtual object of the target object by using the virtual object generation model based on the target voice; The target video is sent to the terminal so that the terminal plays the target video to display the target virtual object.

16. The virtual object display method according to claim 15, characterized in that: The preset object generation model includes a preset vocalization area generation model and a preset face generation model; the preset object generation model is trained based on the follow-up video and the posture video to obtain a virtual object generation model corresponding to the target object, including: Based on the follow-up video, the preset vocalization area generation model is trained to obtain a vocalization area generation model; According to the posture video, the preset facial generation model is trained to obtain a facial generation model; The vocalization area generation model and the facial generation model are used as virtual object generation models corresponding to the target object.

17. The virtual object display method according to claim 16, characterized in that: The follow-up video includes a plurality of follow-up video frames and audio frames corresponding to the follow-up video frames; The method of training the preset vocalization area generation model based on the follow-up reading video to obtain the vocalization area generation model includes: Recognizing a facial image of the target object in the follow-up video frame, and detecting a vocalization area of ​​the target object in the facial image to obtain at least one vocalization area image; Acquire at least one vocalization key point corresponding to the vocalization area image, and construct a vocalization area information set based on the vocalization key point; Target audio data corresponding to the vocalization area is extracted from the audio frame, and the preset vocalization area generation model is trained based on the vocalization area information set and the target audio data to obtain the vocalization area generation model.

18. The virtual object display method according to claim 16, characterized in that: The step of generating a target video of a target virtual object of the target object by using the virtual object generation model based on the target voice includes: Based on the target speech, using the vocalization area generation model to predict a target vocalization area image of the target object; Acquire an original virtual object video corresponding to the target object, and identify target facial information of the original virtual object in the original virtual object video; Based on the target facial information and the target vocalization area image, the facial generation model is used to perform facial correction on the target object to obtain a corrected facial image, and based on the corrected facial image, a target video of a target virtual object of the target object is constructed.

19. A virtual object generation device, characterized in that: include: A display unit, configured to display a virtual object generation page, wherein the virtual object generation page includes an object generation control; A recording unit, configured to record a plurality of videos of the target object in different states in response to a triggering operation of a target object on the object generation control, to obtain an object video, wherein the object video includes a follow-up video and a posture video; The first display unit is used to display a target virtual object corresponding to the target object when a target voice of the target object is received, wherein the target virtual object includes a virtual object generated based on the target voice, the follow-up video and the gesture video.

20. A virtual object generation device, characterized in that: include: A receiving unit, configured to receive an object video of a target object sent by a terminal, wherein the object video includes a follow-up video and a posture video; A training unit, used to train a preset object generation model based on the follow-up video and the posture video to obtain a virtual object generation model corresponding to the target object; a generating unit, configured to generate a target video of a target virtual object of the target object by using the virtual object generation model based on the target voice when receiving the target voice of the target object sent by the terminal; The second display unit is used to send the target video to the terminal so that the terminal plays the target video to display the target virtual object.

21. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the virtual object display method according to any one of claims 1 to 18.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the virtual object display method according to any one of claims 1 to 18.

23. A computer program product, characterized in that The computer program product stores a computer program, and the computer program is suitable for being loaded by a processor to execute the virtual object display method according to any one of claims 1 to 18.

Images