Interaction method and device, electronic equipment and storage medium

Abstract

The invention relates to an interaction method and device, electronic equipment and a storage medium, and the method comprises the steps: displaying a first page which comprises a digital image display window; the digital image display window comprises a target image; the target image comprises a target digital image body; the target digital image body has a digital character corresponding to the target digital image body; dialogue input information is obtained; obtaining a target video based on the dialogue input information and the digital role of the target digital image body; the target video is a speaking video of the target digital image body; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information; and playing the target video in the digital image display window. The essence of the invention is to provide a method for dynamically interacting a digital image body with a user. In the interaction process, the digital image body is dynamic, for example, the digital image body has face and / or limb actions during speaking, and the speaked words conform to the role, so that the vivid feeling of interaction between the digital image body and the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an interaction method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of science and technology, artificial intelligence has become one of the most influential technological fields in the world today. From the early simple machine learning algorithms to the widespread application of deep neural networks today, artificial intelligence has achieved remarkable achievements in many aspects such as image recognition, speech processing, and natural language understanding.

[0003] However, in the field of digital human interaction, although digital human technology has come to the fore, when digital humans interact with users, they are mostly presented in a static manner and lack real-life responses, making the interaction between users and digital humans unnatural. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an interaction method, an apparatus, an electronic device and a storage medium.

[0005] In a first aspect, the present disclosure provides an interaction method, comprising:

[0006] Displaying a first page, wherein the first page includes a digital avatar display window; the digital avatar display window includes a target image; the target image includes a target digital avatar; the target digital avatar has a digital role corresponding thereto;

[0007] Get the dialogue input information;

[0008] Based on the dialogue input information and the digital character of the target digital image, a target video is obtained; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information;

[0009] The target video is played in the digital image display window.

[0010] In a second aspect, the present disclosure further provides an interactive device, including:

[0011] A first display module is used to display a first page, wherein the first page includes a digital image display window; the digital image display window includes a target image; the target image includes a target digital image; the target digital image has a digital role corresponding to it;

[0012] The acquisition module is used to obtain the dialogue input information;

[0013] A video production module, for obtaining a target video based on the dialogue input information and the digital character of the target digital image; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information;

[0014] The second display module is used to play the target video in the digital image display window.

[0015] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] A storage device for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method as described above.

[0019] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned interaction method when executed by a processor.

[0020] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0021] The technical solution provided by the embodiment of the present disclosure is provided by setting a first page for display, the first page includes a digital image display window; the digital image display window includes a target image; the target image includes a target digital image; the target digital image has a digital role corresponding to it; dialogue input information is obtained; based on the dialogue input information and the digital role of the target digital image, a target video is obtained; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information; and the target video is played in the digital image display window. Its essence is to provide a method for a digital image to interact with a user in a dynamic manner. During the interaction process, the digital image is dynamic, such as having facial and / or body movements when speaking, and the words spoken are consistent with its role, which improves the realism of the interaction between the digital image and the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 A flowchart of an interactive method provided by an embodiment of the present disclosure;

[0025] Figure 2-Figure 8 Schematic diagrams of several terminal interfaces provided by embodiments of the present disclosure;

[0026] Fig. 9 is a schematic diagram of the structure of an interactive device in an embodiment of the present disclosure;

[0027] Fig.10 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] Figure 1 This is a flowchart of an interaction method provided by an embodiment of the present disclosure. This embodiment is applicable to situations where a client interacts with a digital image. This method can be executed by an interaction device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a terminal, specifically including but not limited to a smart phone, a PDA, a tablet computer, a wearable device with a display screen, a desktop computer, a laptop computer, an all-in-one machine, a smart home device, etc. Alternatively, this embodiment can be applicable to situations where a server interacts with a digital image. This method can be executed by an interaction device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server.

[0031] like Figure 1 As shown, the method may specifically include:

[0032] S110, displaying a first page, the first page including a digital avatar display window; the digital avatar display window including a target image; the target image including a target digital avatar image; the target digital avatar has a digital role corresponding thereto.

[0033] A digital image is a digital entity representation constructed through a variety of means such as computer graphics technology, artificial intelligence algorithms, and multimedia data processing. It has specific appearance features, can simulate the appearance of humans, animals, or objects, has behavioral characteristics such as facial expressions and / or body movements, and can understand natural language and respond accordingly. In some scenarios, digital images can also make autonomous decisions and complete specific tasks. It can use various interactive interfaces to interact with users in multiple dimensions such as vision, hearing, and even touch, and can be widely used in film and television entertainment, digital marketing, virtual social networking, education and teaching, intelligent customer service, and other fields.

[0034] A digital role may be, for example, a limiting condition for a digital image. Specifically, a digital role may be, for example, a limitation on one or more dimensions of a digital image's personality, environment, and behavior pattern. Different digital images have different digital roles. When the same natural language is input to different digital images, the responses made by different digital images to the same natural language may be different. The response made by any digital image to the input natural language is consistent with its role characteristics. In practice, a digital role may be a person or animal with a real basis (such as a person in history, or a person in a film, television, or literary work), or a purely fictional person or animal with no real basis.

[0035] For example, suppose there are two digital images. One digital image has a digital character: a wise elder, who is calm and wise, and is in a quiet study. The other digital image has a digital character: an elf character, who is in a fantasy forest environment. Suppose when these two digital images are asked questions about the philosophy of life, the answer given by the first digital image is profound and meaningful, while the answer given by the second digital image is smart, playful and imaginative.

[0036] The target image includes a target digital avatar, which may refer to, for example, a digital avatar that the user wishes to communicate with, displayed in an image format. In some scenarios, the target digital avatar may be randomly selected from a plurality of digital avatars, or selected from a plurality of digital avatars that meet preset conditions (such as the most popular, or the one that the user has used in the past, or the one that the user has used most frequently in the past), or selected by the user himself from a plurality of digital avatars.

[0037] The first page may be, for example, a page that provides a user with a dialogue service with a target digital image. The digital image display window may be, for example, a window in the first page that is used to display the target digital image. In practice, the target digital image may be displayed in the form of a static image, a dynamic image, or a video. The target image may be, for example, an image that includes the target digital image, which may be a static image or a dynamic image.

[0038] S120: Acquire dialogue input information.

[0039] The dialogue input information may be, for example, information input by the user for dialogue with the target digital image. In practice, the dialogue input information may be, for example, questions asked to the target digital image, or comments on information previously spoken by the target digital image.

[0040] In practice, the dialogue input information may be input in the form of text or in the form of audio.

[0041] S130, obtaining a target video based on the dialogue input information and the digital character of the target digital image; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information.

[0042] The feedback information may be, for example, a verbal response made by the target digital image to the dialogue input information. For example, if the dialogue input information is a question, the feedback information is the answer to the question expressed in words. If the dialogue input information is a statement of an event, the feedback information is a comment on the event expressed in words.

[0043] The content of the target video is a video of the target digital avatar speaking feedback information. In the target video, the target digital avatar speaks feedback information accompanied by the movement of the target digital avatar's facial (including mouth) muscles, and even body movements. Moreover, the movement of the target digital avatar (including facial movement, and even body movement) is synchronized with the feedback information, so as to give the user the impression that the feedback information heard is indeed spoken by the target digital avatar.

[0044] There are many methods for implementing this step, and this application does not limit this. Exemplarily, the target digital image has a language processing model corresponding to it, and the language processing model has the function of outputting feedback information of the digital character that conforms to the digital image based on the dialogue input information; the implementation method of this step may include: inputting the dialogue input information into the language processing model corresponding to the target digital image to obtain feedback information; processing the feedback information into audio information; and obtaining a target video based on the feedback information, the audio information, and the target image, in which the voice and action of the digital image are synchronized.

[0045] In practice, the digital image and the language processing model can be set to correspond one to one. The language processing model is configured or trained by the digital character of the digital image corresponding to it, and the feedback information output by the language processing model is consistent with the digital character of the digital image corresponding to it.

[0046] In practice, if the dialogue input information is text information, the dialogue input information can be directly input into the language processing model corresponding to the target digital image to obtain feedback information. If the dialogue input information is audio information, the dialogue input information can be first converted into audio information, and then the audio information can be input into the language processing model corresponding to the target digital image to obtain feedback information. The feedback information directly output by the language processing model can be text information.

[0047] The voice and action synchronization of the digital avatar in the target video may include, for example, the voice and lip synchronization of the digital avatar in the target video. Further, the voice and body movement synchronization of the digital avatar in the target video may be arranged.

[0048] Furthermore, there are many methods for "obtaining a target video based on feedback information and a target image", and this application does not limit this. Exemplarily, in one embodiment, "obtaining a target video based on feedback information, audio information, and a target image" includes: adjusting the facial image and / or body image of the target digital image in the target image based on the feedback information to obtain an adjusted target image; synthesizing the audio information and the adjusted target image to obtain a target video.

[0049] Since the target video contains multiple image frames arranged in a certain order, and each image frame is obtained by adjusting the facial image and / or limb image of the target digital image in the target image based on feedback information, this means that the backgrounds of any two image frames in the target video are the same or similar.

[0050] In another embodiment, "obtaining a target video based on feedback information, audio information and a target image" may include: generating a candidate video based on the target image; the candidate video includes a target digital image; the candidate video includes multiple image frames, at least some of the image frames have different backgrounds; based on the feedback information, adjusting the facial image and / or limb image of the target digital image in the image frame of the candidate video to obtain an adjusted candidate video; synthesizing the audio information and the adjusted candidate video to obtain the target video.

[0051] The candidate video may be, for example, a video generated by a video generation model with a target image as a guide image. In some scenarios, each image frame of the candidate video may be set to be a target digital image; or some image frames include the target digital image, and some image frames do not include the target digital image. In the image frames of the candidate video, the rest of the image frames except the target digital image are the background. The backgrounds of at least some image frames in the candidate video are different. The candidate video is generated based on the target image, wherein the backgrounds of at least some image frames of the candidate video are different. The audio information is synthesized with the candidate video in which the facial image and / or body image of the target digital image is adjusted to obtain the target video. The target video can simulate the target digital image to make a relatively large movement, such as simulating the target digital image to enter another room from one room, or simulating the rapid passing of objects behind the target digital image when the target digital image moves.

[0052] Furthermore, there are many specific implementation methods for "processing feedback information into audio information", and this application does not limit this. Exemplarily, processing feedback information into audio information includes: based on the digital character of the target digital image, processing the feedback information into audio information so that the timbre of the audio information corresponds to the digital character of the target digital image. Exemplarily, if the digital character of the target digital image is a wise elder, the timbre of the audio information obtained from the feedback information is an old timbre, which is consistent with the character characteristics of the wise elder. The purpose of this setting is to match the audio information of the target video obtained subsequently with the digital character of the target digital image.

[0053] S140: Play the target video in the digital image display window.

[0054] For example, Figure 2 and Figure 3 Schematic diagrams of two first pages provided in the embodiments of the present disclosure. Figure 2 The first page in can be applied to the APP side, Figure 3 The first page in can be applied to the web. See Figure 2 or Figure 3 , the first page displays a digital image display window, and the digital image display window displays a target image including a target digital image, and the target digital image is a lady. Assume that the digital character of the lady is a doctor with a calm personality and rich medical expertise. If the user inputs dialogue input information in the form of language or text, such as "My face has turned yellow recently." In the target video played in the digital image display window, the target digital image responds: "Have you eaten a lot of carrots, pumpkins, tangerines, and oranges recently? If so, as long as you stop eating these foods for a period of time, your skin will gradually return to normal, don't worry."

[0055] The above technical solution is provided by setting a first page for display, the first page includes a digital image display window; the digital image display window includes a target image; the target image includes a target digital image; the target digital image has a digital role corresponding to it; dialogue input information is obtained; based on the dialogue input information and the digital role of the target digital image, a target video is obtained; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information; and the target video is played in the digital image display window. Its essence is to provide a method for a digital image to interact with a user in a dynamic manner. During the interaction process, the digital image is dynamic, such as having facial and / or body movements when speaking, and the words spoken are consistent with its role, which improves the realism of the interaction between the digital image and the user.

[0056] On the basis of the above technical solution, optionally, the first page also includes a dialogue display window; the method also includes: displaying dialogue input information and feedback information in the dialogue display window.

[0057] The dialog display window may be, for example, an area in the first page for displaying dialog input information and feedback information.

[0058] For example, see Figure 3 The first page includes a dialogue display window, in which dialogue input information and feedback information are displayed in the form of message bubbles.

[0059] By displaying the dialogue input information and feedback information in the dialogue display window, users can understand and review the communication content with the target digital image.

[0060] Furthermore, the first page also includes a dialogue input option. The display position of the dialogue input option can be located in the digital image display window or in the dialogue display window. Figure 3 In the dialog input options, the "Press to start voice input" option and the "Text input" option are both dialog input options. If the user triggers "Press to start voice input" (such as clicking, long pressing, or hovering), the user's voice is recorded and the recorded user's voice is used as dialog input information. If the user triggers "Text input" (such as clicking, long pressing, or hovering), the text input box is displayed and the user can enter text information in the text input box. The text information entered by the user in the text input box is used as dialog input information.

[0061] Based on the above technical solutions, optionally, the method may also include: acquiring a user image in a mode of a video call with a target digital image; and displaying the user image in a call end display window on the first page.

[0062] For example, see Figure 3 The first page includes a "video call" option. If the user triggers the "video call" option (such as clicking, long pressing, or hovering, etc.), the user enters a video call mode with the target digital image. Figure 4 In this mode, the electronic device uses the camera to collect images of the user, obtains the user image, and displays the user image in the call end display window of the first page. This can simulate the scene of the user making a video call with the target digital image, giving the user an immersive and realistic feeling.

[0063] Based on the above technical solution, optionally, S110 may include: displaying a digital avatar display page; the digital avatar display page includes multiple digital avatar identifiers; different digital avatars have different digital roles; in response to a selection operation on a target digital avatar identifier in the digital avatar display page, displaying a first page.

[0064] The digital avatar display page may be, for example, a page for displaying a digital avatar identifier. Optionally, the digital avatar display page includes a plurality of digital avatar identifiers. The digital avatar identifier may be, for example, a page for distinguishing a digital avatar from other digital avatars. Exemplarily, the digital avatar identifier includes at least one of the following: a digital avatar image, a digital avatar name, and digital character description information of the digital avatar.

[0065] When the digital avatar display page includes multiple digital avatar identifiers, the digital avatar indicated by the digital avatar identifier selected by the user is the target digital avatar. The selection operation on the target digital avatar identifier may be, for example, a click operation or a drag operation on the target digital avatar identifier.

[0066] For example, see Figure 5 The digital image display page includes multiple digital image logos. If the user clicks on the digital image 2 logo, see Figure 3 , displaying the first page. In the first page, the digital image display window displays the target image including the digital image 2.

[0067] By setting up a digital avatar display page; the digital avatar display page includes multiple digital avatar identifiers; different digital avatars have different digital roles; in response to the selection operation of the target digital avatar identifier in the digital avatar display page, the first page is displayed. In essence, the digital avatar display page provides users with multiple digital avatars, so that users can choose and use digital avatars that meet their needs according to their own needs, so as to meet the user's use needs in specific scenarios such as social, expression and work.

[0068] In actual application scenarios, many digital avatars can be classified according to various standards. Each category has a label that matches it. The digital avatar display page includes multiple labels. When a label is selected, the digital avatar display page displays the digital avatar corresponding to the label. For example, see Figure 5 , in this page, "Character Plaza", "My Character", "All", "Realistic", and "Second Dimension" are all labels. If "Character Plaza" and "All" are both selected, the identification information of all digital characters created by different users will be displayed on the digital character display page. If "Character Plaza" and "Realistic" are both selected, the identification information of realistic digital characters created by different users will be displayed on the digital character display page. If "My Character" and "All" are both selected, the identification information of all digital characters created by the same user will be displayed on the digital character display page. If "My Character" and "Realistic" are both selected, the identification information of realistic digital characters created by the same user will be displayed on the digital character display page.

[0069] Based on the above technical solution, optionally, the method may also include: displaying a second page; the second page includes a digital avatar creation option; in response to a selection operation on the digital avatar creation option, displaying a digital avatar creation page; collecting digital avatar materials and digital character description information of the digital avatar through the digital avatar creation page; the digital avatar materials include a video or image of the digital avatar; based on the digital avatar materials and the character description information, determining a target image including the digital avatar, and configuring a corresponding language processing model for the digital avatar.

[0070] The second page may be, for example, a page including digital avatar creation options. In some scenarios, the second page may be a digital avatar display page.

[0071] The digital avatar creation option may be, for example, an option that can guide the user to enter a digital avatar creation page. The selection operation on the digital avatar creation option may be, for example, a click operation, a drag operation, or a slide operation on the digital avatar creation option.

[0072] The digital avatar material can be, for example, the basic material used to construct the digital avatar, and is used to define the specific form of the digital avatar. It can be presented in the form of a video or an image. For example, if a user wants to create a digital avatar with the appearance of a kitten, an image with a kitten as the main body or a video with a kitten as the main body can be input.

[0073] The digital character description information may include, for example, the character and / or environment of the digital image.

[0074] For example, see Figure 5 , because this page includes the option of "Create My Character", and this option of "Create My Character" belongs to the option of creating a digital image, this page can also be regarded as the second page. If the user clicks on the option of "Create My Character", see Figure 6 , showing the digital image creation page. In this design, digital image creation is divided into three steps. Step 1, making requirements; Step 2, making character images; Step 3, supplementing character information. Figure 6 The digital image creation page in corresponds to step one and is used to help users understand the production requirements. Figure 7 The digital image creation page in corresponds to step 2 and is used to collect digital image materials. Figure 8 The digital image creation page in corresponds to step three and is used to collect digital character description information.

[0075] The digital avatar creation page collects digital avatar materials (including videos or images of digital avatars) and digital character description information of digital avatars. Based on the materials and character description information, the target image containing the digital avatar is determined, and the corresponding language processing model is configured for the digital avatar. Its essence is to allow users to set up digital avatars according to their own needs to meet their personalized usage needs.

[0076] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0077] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0078] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0079] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0080] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0081] Fig. 9 Schematic diagram of the structure of an interactive device in an embodiment of the present disclosure. The interactive device provided in the embodiment of the present disclosure can be configured in a client, or can be configured in a server. Fig. 9 , the interactive device specifically includes:

[0082] The first display module 310 is used to display a first page, wherein the first page includes a digital avatar display window; the digital avatar display window includes a target image; the target image includes a target digital avatar; the target digital avatar has a digital character corresponding to it;

[0083] An acquisition module 320, used to acquire dialogue input information;

[0084] The video production module 330 is used to obtain a target video based on the dialogue input information and the digital character of the target digital image; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information;

[0085] The second display module 340 is used to play the target video in the digital image display window.

[0086] Furthermore, the target digital image has a language processing model corresponding thereto, and the language processing model has the function of outputting feedback information of the digital character that conforms to the digital image based on the dialogue input information; the video production module 330 is used to:

[0087] Inputting the dialogue input information into a language processing model corresponding to the target digital image to obtain feedback information;

[0088] Processing the feedback information into audio information;

[0089] Based on the feedback information, the audio information and the target image, a target video is obtained, in which the voice and action of the digital image are synchronized.

[0090] Furthermore, the video production module 330 is used to:

[0091] Based on the feedback information, adjusting the facial image and / or limb image of the target digital image in the target image to obtain an adjusted target image;

[0092] The audio information and the adjusted target image are synthesized to obtain a target video.

[0093] Furthermore, the video production module 330 is used to:

[0094] Based on the target image, a candidate video is generated; the candidate video includes the target digital image; the candidate video includes a plurality of image frames, at least some of the image frames have different backgrounds;

[0095] Based on the feedback information, adjusting the facial image and / or body image of the target digital image in the image frame of the candidate video to obtain an adjusted candidate video;

[0096] The audio information and the adjusted candidate video are synthesized to obtain a target video.

[0097] Furthermore, the video production module 330 is used to:

[0098] Based on the digital character of the target digital avatar, the feedback information is processed into audio information so that the timbre of the audio information corresponds to the digital character of the target digital avatar.

[0099] Furthermore, the device also includes a third display module, which is used to:

[0100] The dialogue input information and the feedback information are displayed in the dialogue display window.

[0101] Furthermore, the device also includes a fourth display module, which is used to:

[0102] Acquiring a user image in a video call mode with the target digital avatar;

[0103] The user image is displayed in a call end display window of the first page.

[0104] Furthermore, the first display module 310 is used to:

[0105] Displaying a digital avatar display page; the digital avatar display page includes a plurality of digital avatar identifiers; the digital roles of different digital avatars are different;

[0106] In response to a selection operation of a target digital avatar identifier in the digital avatar display page, the first page is displayed.

[0107] Furthermore, the device also includes a creation module, which is used to:

[0108] Displaying a second page; the second page includes a digital avatar creation option;

[0109] In response to a selection operation of the digital avatar creation option, displaying a digital avatar creation page;

[0110] Collect digital avatar materials and digital character description information of the digital avatar through the digital avatar creation page; the digital avatar materials include videos or images of the digital avatar;

[0111] Based on the digital avatar material and the role description information, a target image including the digital avatar is determined, and a language processing model corresponding to the digital avatar is configured for the digital avatar.

[0112] The interactive device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the interactive method provided in the embodiment of the method of the present disclosure, and has execution steps and beneficial effects, which will not be repeated here.

[0113] Fig.10 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Fig.10 , which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Fig.10 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0114] like Fig.10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the interactive method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0115] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Fig.10 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0116] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the interactive method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0117] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0118] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0119] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0120] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0121] Displaying a first page, wherein the first page includes a digital avatar display window; the digital avatar display window includes a target image; the target image includes a target digital avatar; the target digital avatar has a digital role corresponding thereto;

[0122] Get the dialogue input information;

[0123] Based on the dialogue input information and the digital character of the target digital image, a target video is obtained; the target video is a speaking video of the target digital image; in the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information;

[0124] The target video is played in the digital image display window.

[0125] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0126] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0127] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0129] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0132] one or more processors;

[0133] A memory for storing one or more programs;

[0134] When the one or more programs are executed by the one or more processors, the one or more processors implement any interaction method provided in the present disclosure.

[0135] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the program implements any of the interaction methods provided by the present disclosure.

[0136] The embodiments of the present disclosure further provide a computer program product, which includes a computer program or instructions, and the computer program or instructions implement the above-mentioned interaction method when executed by a processor.

[0137] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0138] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interactive method, characterized in that: include: Displaying a first page, wherein the first page includes a digital avatar display window; The digital image display window includes a target image; the target image includes a target digital image; the target digital image has a digital role corresponding to it; Get the dialogue input information; Based on the dialogue input information and the digital character of the target digital image, a target video is obtained; The target video is a speaking video of the target digital image; In the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information; The target video is played in the digital image display window.

2. The method according to claim 1, characterized in that The target digital image has a language processing model corresponding thereto, and the language processing model has the function of outputting feedback information of the digital character that conforms to the digital image based on the dialogue input information; The step of obtaining a target video based on the dialogue input information and the digital character of the target digital image includes: Inputting the dialogue input information into a language processing model corresponding to the target digital image to obtain feedback information; Processing the feedback information into audio information; Based on the feedback information, the audio information and the target image, a target video is obtained, in which the voice and action of the digital image are synchronized.

3. The method according to claim 2, characterized in that The obtaining of a target video based on the feedback information, the audio information and the target image comprises: Based on the feedback information, adjusting the facial image and / or limb image of the target digital image in the target image to obtain an adjusted target image; The audio information and the adjusted target image are synthesized to obtain a target video.

4. The method according to claim 2, characterized in that: The obtaining of a target video based on the feedback information, the audio information and the target image comprises: Based on the target image, a candidate video is generated; the candidate video includes the target digital image; the candidate video includes a plurality of image frames, at least some of the image frames have different backgrounds; Based on the feedback information, adjusting the facial image and / or body image of the target digital image in the image frame of the candidate video to obtain an adjusted candidate video; The audio information and the adjusted candidate video are synthesized to obtain a target video.

5. The method according to claim 2, characterized in that: The processing of the feedback information into audio information comprises: Based on the digital character of the target digital avatar, the feedback information is processed into audio information so that the timbre of the audio information corresponds to the digital character of the target digital avatar.

6. The method according to claim 1, characterized in that The first page also includes a dialog display window; and further includes: The dialogue input information and the feedback information are displayed in the dialogue display window.

7. The method according to claim 1, characterized in that Also includes: Acquiring a user image in a video call mode with the target digital avatar; The user image is displayed in a call end display window of the first page.

8. The method according to claim 1, characterized in that The display of the first page further includes: Displaying a digital avatar display page; the digital avatar display page includes a plurality of digital avatar identifiers; the digital roles of different digital avatars are different; In response to a selection operation of a target digital avatar identifier in the digital avatar display page, the first page is displayed.

9. The method according to claim 1, characterized in that: Also includes: Displaying a second page; the second page includes a digital avatar creation option; In response to a selection operation of the digital avatar creation option, displaying a digital avatar creation page; Collect digital avatar materials and digital character description information of the digital avatar through the digital avatar creation page; the digital avatar materials include videos or images of the digital avatar; Based on the digital avatar material and the role description information, a target image including the digital avatar is determined, and a language processing model corresponding to the digital avatar is configured for the digital avatar.

10. An interactive device, characterized in that: include: A first display module, used to display a first page, wherein the first page includes a digital avatar display window; The digital image display window includes a target image; the target image includes a target digital image; the target digital image has a digital role corresponding to it; The acquisition module is used to obtain the dialogue input information; A video production module, used for obtaining a target video based on the dialogue input information and the digital character of the target digital image; The target video is a speaking video of the target digital image; In the target video, the words spoken by the target digital image are feedback information corresponding to the dialogue input information; The second display module is used to play the target video in the digital image display window.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images