Dialogue processing method and device and related product

Abstract

Embodiments of the invention provide a dialogue processing method and apparatus, and a related product. The method comprises the steps of obtaining video data generated for a first role in a first application program; the first role is created based on a generative model; the first role in the first application program is used for dialogue with a dialogue object; the first role in the video data executes an action matched with the pronunciation content and outputs the pronunciation content; the pronunciation content is related to dialogue content, and the dialogue content is dialogue content between the first role and the dialogue object; and displaying the video data in the first application program.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a dialogue processing method, device and related products. Background Art

[0002] Currently, characters can be created through generative models, and users can talk to characters to complete story chapters or perform story tasks. In the prior art, users talk to characters through text or voice. If a wonderful conversation occurs between the user and the character, the user can only record the wonderful conversation manually, which is inefficient for users to record wonderful conversations. Summary of the invention

[0003] The disclosed embodiments provide a conversation processing method, device and related products, which can obtain and display video data of a first character, record the wonderful conversation between the first character and the conversation partner through the video data of the first character, and improve the efficiency of recording the wonderful conversation.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for processing a conversation, including: Obtain video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The video data is presented in the first application.

[0005] In a second aspect, an embodiment of the present disclosure provides a dialog processing device, including: A data acquisition unit is used to acquire video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; A data display unit is used to display the video data in the first application.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In one or more embodiments of the present disclosure, video data generated for a first character in a first application can be obtained and displayed, wherein the first character is created based on a generative model, the first character in the first application is used to have a conversation with a dialogue object, the first character in the video data performs an action that matches the pronunciation content and outputs the pronunciation content, and the pronunciation content is related to the conversation content between the first character and the dialogue object. It can be seen that through this embodiment, in a scenario where the dialogue object and the first character in the first application have a conversation, the pronunciation content can be determined according to the conversation content between the first character and the dialogue object and the video data of the first character can be generated, and the first character in the video data performs an action that matches the pronunciation content and outputs the pronunciation content, thereby achieving the effect of recording the pronunciation content through the video data, that is, achieving the effect of recording the wonderful conversation between the first character and the dialogue object through the video data, and improving the efficiency of recording the wonderful conversation. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. Figure 1 A flowchart of a method for processing a conversation provided by an embodiment of the present disclosure; Figure 2a A schematic diagram of starting a first application provided by an embodiment of the present disclosure; Figure 2b A schematic diagram of video data of a first character provided by an embodiment of the present disclosure; Figure 3 A schematic diagram of a process for generating and displaying video data of a first character provided by an embodiment of the present disclosure; Figure 4 A schematic diagram of a process of generating and displaying video data of a first character provided by another embodiment of the present disclosure; Figure 5A schematic diagram of the structure of a dialog processing device provided by an embodiment of the present disclosure; Figure 6 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present disclosure.

[0012] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0013] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0014] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0015] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0016] The disclosed embodiment provides a dialogue processing method, which can obtain and display video data of a first character, record the wonderful dialogue between the first character and the dialogue object through the video data of the first character, and improve the efficiency of recording the wonderful dialogue.

[0017] Figure 1 A flow chart of a method for processing a conversation provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the process includes: Step S102, obtaining video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to the conversation content, and the conversation content is the conversation content between the first character and the dialogue partner; Step S104: display the video data in the first application.

[0018] In the disclosed embodiment, video data generated for a first character in a first application can be obtained and displayed, wherein the first character is created based on a generative model, the first character in the first application is used to have a conversation with a dialogue object, the first character in the video data performs an action that matches the pronunciation content and outputs the pronunciation content, and the pronunciation content is related to the conversation content between the first character and the dialogue object. It can be seen that through this embodiment, in a scenario where the dialogue object and the first character in the first application have a conversation, the pronunciation content can be determined according to the conversation content between the first character and the dialogue object and the video data of the first character can be generated, and the first character in the video data performs an action that matches the pronunciation content and outputs the pronunciation content, thereby achieving the effect of recording the pronunciation content through the video data, that is, achieving the effect of recording the wonderful conversation between the first character and the dialogue object through the video data, and improving the efficiency of recording the wonderful conversation.

[0019] Figure 1 The method process can be executed by a terminal device or a server, and the terminal device includes but is not limited to mobile phones, computers, laptops, tablet computers, car computers, wearable devices and other electronic products. The server includes but is not limited to a single server or a server cluster, and the server can be the background server of the first application.

[0020] In the above step S102, video data generated for the first character in the first application is obtained. In this embodiment, the first application may have at least one of the following two functions: 1. For a character created and published by a user in the first application, the character is created through a generative model such as a large language model LLM (Large Language Model), and a page for chatting with the character is provided to the user to achieve interaction between the user and the character; 2. For a story created and published by a user in the first application, one or more characters in the story are created through a generative model, and a page for chatting with the created character is provided to the user, so that the user participates in the story process and completes the story task.

[0021] Based on the function of the first application, the first application has a first role, the first role is created based on the generative model, and the first role can be a role in a story published by a user in the first application, or a role independent of the story published by the user in the first application. In this embodiment, in the first application, the first role is used to communicate with a dialogue object, and the dialogue object can be the above-mentioned user.

[0022] In the video data of the first character, the first character outputs the pronunciation content and performs actions that match the pronunciation content. For example, the first character says "good morning", and the mouth shape, facial expression and gestures also match "good morning". In this embodiment, in the video data of the first character, the pronunciation content of the first character is related to the conversation content of the first character, and the conversation content is the conversation content between the first character and the dialogue object. For example, the first character is discussing the topic of "aerial photography" with the user, then in the video data of the first character, the pronunciation content of the first character can be "I really want to use a drone to shoot the scenery", and for another example, the first character is discussing the topic of "XX singer" with the user, then in the video data of the first character, the pronunciation content of the first character can be singing the representative song of the singer.

[0023] In this embodiment, after the video data generated for the first character in the first application is obtained, the video data is displayed in the first application in step S104. For example, the video data is displayed in a chat page between the first character and a dialogue object such as a user in the first application, so that during the dialogue between the first character and the user, the video data of the first character is synchronously displayed to help the user record the wonderful dialogue between the first character and the user, thereby improving the efficiency of recording the wonderful dialogue.

[0024] It is worth noting that the dialogue object involved in each embodiment of the present disclosure can be the user of the terminal device, or any party operating the terminal device, such as an automatically executed program operating the terminal device or a robot operating the terminal device.

[0025] In one embodiment, the method further comprises the following steps for generating video data of the first character: Acquire a first image of a first character, and acquire pronunciation content; generating video data according to the first image and the pronunciation content; The first character has a first image in the first image; the first character has a second image in the video data; and the second image matches the first image.

[0026] In this embodiment, first, a first image of the first character is obtained, which may be a static image, and the above-mentioned pronunciation content is obtained, which is the pronunciation content to be output by the first character in the video data of the first character. Then, the video data of the first character is generated according to the first image and the pronunciation content.

[0027] In this embodiment, the first character in the first image has a first image, and the first image includes but is not limited to the appearance, clothing, and actions of the first character. The first character in the video data has a second image, and the second image includes but is not limited to the appearance, clothing, and actions of the first character, and the second image matches the first image, so that the first character can output the above-mentioned pronunciation content and perform actions matching the pronunciation content based on the image in the first image, thereby obtaining the video data of the first character.

[0028] In one example, based on the principle of generating live photos, the first character in the first image can be driven to output the pronunciation content and perform matching actions such as controlling mouth movements to achieve a dynamic speaking effect and obtain video data of the first character.

[0029] It can be seen that through this embodiment, the first image of the first character can be obtained, as well as the above-mentioned pronunciation content. According to the first image and the pronunciation content, the video data of the first character can be accurately generated, and the image of the first character in the first image is matched with the image of the first character in the video data, so as to achieve the effect of driving the character to speak and generate video data without changing the character's image.

[0030] In one embodiment, obtaining a first image of a first character includes: Acquire a conversation background between a first character and a conversation object in a first application as a first image; the conversation background is used to represent the first character; or, An image of a first character specified by a dialog object is acquired as a first image.

[0031] In this embodiment, in one case, in a chat page between a first character and a dialogue object provided by a first application, there is a dialogue background between the first character and the dialogue object, and the dialogue background may be a static image used to represent the first character. Then, based on user settings or system pre-settings, the dialogue background may be used as the first image of the first character, and video data of the first character may be generated based on the dialogue background. The dialogue object, such as a user, may set the dialogue background to the first image of the first character by triggering the dialogue background.

[0032] In another case of this embodiment, the dialogue object, such as the user, can customize the image based on which the video data of the first character is generated. Therefore, the dialogue object, such as the user, can specify the first image of the first character, based on which, the image of the first character specified by the dialogue object is obtained and used as the first image of the first character. The dialogue object can implement the process of specifying the first image by uploading the first image to the first application.

[0033] It can be seen that through this embodiment, different first images can be obtained in different situations, so that the system can pre-generate video data of the first character or the user can customize the video data of the first character.

[0034] In one embodiment, obtaining the above-mentioned pronunciation content includes: Acquire a first dialogue message sent by the first character to the dialogue object as a target dialogue message, or acquire a second dialogue message between the first character designated by the dialogue object and the dialogue object as a target dialogue message; Generate pronunciation content based on the target conversation message.

[0035] In this embodiment, when obtaining the pronunciation content of the first character in the video data, the first dialogue message sent by the first character to the dialogue object can be obtained as the target dialogue message, and the first dialogue message can be a text message, a voice message, or an image message. Alternatively, a second dialogue message between the first character specified by the dialogue object and the dialogue object is obtained as the target dialogue message. The second dialogue message is specified by the dialogue object such as the user, and the second dialogue message can be a text message, a voice message, or an image message sent by the first character to the dialogue object, or a text message, a voice message, or an image message sent by the dialogue object to the first character.

[0036] In one example, when the video data of the first character is pre-generated by the system, the system can obtain the first conversation message sent by the first character to the dialogue object as the target conversation message. When the video data of the first character is custom-generated by the user, the system can obtain the second conversation message between the first character and the dialogue object specified by the user as the target conversation message.

[0037] Next, the pronunciation content is generated based on the target conversation message. For example, the pronunciation content is generated based on the content of the target conversation message.

[0038] It can be seen that through this embodiment, pronunciation content can be generated based on the target dialogue message between the first character and the dialogue object, so that the pronunciation content of the first character in the video data of the first character is related to the dialogue content between the first character and the dialogue object, thereby achieving the effect of recording the wonderful dialogue between the first character and the dialogue object through video data, and improving the efficiency of recording wonderful dialogues.

[0039] In one embodiment, obtaining a first dialog message sent by a first role to a dialog object as a target dialog message includes: Determine the message heat of each conversation message sent by the first role to the conversation object; According to the message popularity, a first dialog message is determined in each dialog message, and the first dialog message is used as a target dialog message.

[0040] In this embodiment, the message heat of each dialogue message sent by the first role to the dialogue object can be determined, for example, the message heat of each dialogue message sent by the first role to the user can be determined, and the dialogue message can be a text message, a voice message, or an image message. In one example, the message heat of the dialogue message can be determined based on the content of the dialogue object's reply to the dialogue message. For example, if the dialogue object replies to the content of the dialogue message to indicate that the dialogue object is very interested in the content of the dialogue message and wants to continue the dialogue around the content of this dialogue message, the message heat of the dialogue message is determined to be high heat. If the dialogue object replies to the content of the dialogue message to indicate that the dialogue object is not very interested in the content of the dialogue message and does not want to continue the dialogue around the content of this dialogue message, the message heat of the dialogue message is determined to be low heat. Of course, the message heat of the dialogue message can also be determined based on the number of dialogue rounds between the dialogue object and the first role around the content of the dialogue message. For example, if the number of conversation rounds between the conversation object and the first character surrounding the content of the conversation message exceeds 10 times, the message heat of the conversation message is determined to be high heat; if the number of conversation rounds between the conversation object and the first character surrounding the content of the conversation message exceeds 5 times, the message heat of the conversation message is determined to be medium heat; if the number of conversation rounds between the conversation object and the first character surrounding the content of the conversation message is less than 5 times, the message heat of the conversation message is determined to be low heat.

[0041] Next, according to the message heat of each dialogue message sent by the first role to the dialogue object, a dialogue message with a message heat greater than a heat threshold is determined as the first dialogue message among each dialogue message, and the first dialogue message is used as the target dialogue message. For example, a dialogue message with a message heat greater than a low heat is determined as the first dialogue message among each dialogue message, and the first dialogue message is used as the target dialogue message. Of course, the dialogue message with the highest message heat can also be determined as the first dialogue message among each dialogue message.

[0042] In one example, when the system pre-generates video data of the first character, the system can determine the message heat of each conversation message sent by the first character to the conversation object, determine the first conversation message among the conversation messages based on the message heat, and use the first conversation message as the target conversation message.

[0043] It can be seen that through this embodiment, the first conversation message can be determined in each conversation message according to the message heat of each conversation message sent by the first character to the conversation object, and the first conversation message can be used as the target conversation message, thereby achieving the effect of recording the wonderful conversation between the first character and the conversation object through the video data of the first character.

[0044] In one embodiment, after determining that the target dialogue message is obtained, generating pronunciation content based on the target dialogue message includes: Use the content in the target conversation message as the pronunciation content; or, The content associated with the content in the target conversation message is used as the pronunciation content.

[0045] In this embodiment, in one case, the content in the target dialogue message is used as the pronunciation content. For example, if the target dialogue message is a text message "We are friends", or if the target dialogue message is a voice message and the text corresponding to the voice message is "We are friends", then the pronunciation content is "We are friends". In this case, the content of the target dialogue message is the pronunciation content, and in the video data of the first character, the first character is used to output the content of the target dialogue message, so as to achieve the effect of using the first character to record and repeat the target dialogue message.

[0046] In another case, the content associated with the content in the target dialogue message is used as the pronunciation content. For example, if the content in the target dialogue message is related to "XX singer", the pronunciation content can be the representative song of the singer. For another example, if the content in the target dialogue message is related to the topic of "aerial photography", the pronunciation content can be the content related to the topic of aerial photography. In this case, the pronunciation content may not be the content of the dialogue message sent by the first character or the dialogue object. In this case, the pronunciation content is obtained based on the content of the target dialogue message. In the video data of the first character, the first character is used to output the content related to the content in the target dialogue message, so as to achieve the effect of extending the target dialogue message by using the first character.

[0047] It can be seen that through this embodiment, a variety of methods can be used to obtain the pronunciation content according to the content of the target dialogue message, so that in the video data of the first character, by making the first character output the pronunciation content, the wonderful dialogue between the user and the first character is recorded.

[0048] In one embodiment, after acquiring a first image of a first character and acquiring pronunciation content, generating video data according to the first image and the pronunciation content includes: Determine the pronunciation duration corresponding to the pronunciation content through the generative model, and obtain the action requirements of the first character; Through the generative model, the first character in the first image is rendered according to the action requirements, pronunciation content and pronunciation duration to obtain video data; the rendered first character in the video data moves according to the action requirements, pronunciation content and pronunciation duration.

[0049] In this embodiment, video data of the first character is generated according to the first image and pronunciation content through a generative model. The generative model may be a model for creating the first character, or may be another generative model other than the model for creating the first character. The generative model may also be a large language LLM model.

[0050] First, the generative model is used to determine the pronunciation duration corresponding to the pronunciation content. The pronunciation duration corresponding to the pronunciation content is the pronunciation duration required for the first character to output the pronunciation content. For example, if the first character needs 3 seconds to output the pronunciation content, the pronunciation duration is 3 seconds. Also, the generative model is used to obtain the action requirements of the first character. The action requirements of the first character are the requirements for the actions performed when the first character outputs the pronunciation content. For example, the first character is required to match the mouth shape with the pronunciation content when outputting the pronunciation content, and to display facial expressions and gestures that match the pronunciation content. The action requirements of the first character can be pre-configured by the system or set by the user and stored in the generative model.

[0051] Next, the generative model is used to render the first character in the first image according to the action requirements, pronunciation content, and pronunciation duration to obtain video data. Specifically, the first character in the first image can be driven to move according to the action requirements, pronunciation content, and pronunciation duration, thereby obtaining video data of the first character. In one example, a livephoto can be generated to drive the first character in the first image to output the pronunciation content according to the pronunciation duration and perform matching actions such as controlling the mouth shape movement according to the action requirements to obtain video data of the first character.

[0052] Since the video data is obtained by driving the first character in the first image to output the pronunciation content according to the pronunciation duration and perform the matching action according to the action requirements, the background of the first image can be retained in the video data, and the image of the first character in the first image can also be retained, so that the video data is generated while maximally retaining the character image and background, thereby improving the degree of matching between the video data and the content of the first image.

[0053] It can be seen that through this embodiment, the generative model can be used to determine the pronunciation duration corresponding to the pronunciation content, as well as to obtain the action requirements of the first character. Moreover, through the generative model, the first character in the first image is rendered according to the action requirements, pronunciation content and pronunciation duration to obtain video data, and the generative model can be used to accurately and quickly obtain the video data of the first character.

[0054] In one embodiment, obtaining the action requirement of the first character includes: Obtaining the action requirement of the first character specified by the dialogue partner or set by the system; the action requirement of the first character includes at least one of the action part requirement, action angle requirement, and action direction requirement of the first character.

[0055] In this embodiment, the action requirement of the first character can be specified by the dialogue object such as the user, or pre-configured by the system of the terminal device. The system in each embodiment of the present disclosure can be the system of the terminal device. Furthermore, in this embodiment, the action requirement of the first character specified by the dialogue object or set by the system is obtained through a generative model. The action requirement of the first character includes at least one of the action part requirement, action angle requirement, and action direction requirement of the first character. For example, the action requirement of the first character is used to require that the action part of the first character when outputting the pronunciation content includes the face, lips and hands, the action requirement of the first character is used to require that the action angle of the hands when the first character outputs the pronunciation content does not exceed 20 degrees, the face and lip action angles do not exceed 10 degrees, and the action requirement of the first character is used to require that the action direction of the hands when the first character outputs the pronunciation content is toward the outside of the body, and the action direction of the face and lips is not limited.

[0056] It can be seen that through this embodiment, since the action requirement of the first character includes at least one of the action part requirement, action angle requirement, and action direction requirement of the first character, the movement of the first character can be accurately controlled through the action requirement of the first character, so that the movement of the first character in the video data matches the pronunciation content.

[0057] After the video data of the first character is generated and obtained, in the above step S104, the video data is displayed in the first application. In one embodiment, displaying the video data in the first application includes: In response to a trigger event for the first application, the first application is started; the trigger event includes a trigger event for an icon of the first application or a notification message; the content of the notification message is related to the pronunciation content; After the first application is launched, the video data is presented in the first application.

[0058] In this embodiment, the video data of the first character can be generated in advance and displayed after the user enters the first application. When displaying the video data, in one case, the dialogue object such as the user triggers the icon of the first application to start the first program, and the terminal device starts the first application based on the user operation. In another case, the dialogue object such as the user triggers the notification message of the first application, and the content of the notification message is related to the pronunciation content of the first character. For example, the pronunciation content of the first character is "Come and chat with me", then the content of the notification message can be "XX character is looking for you to chat". For another example, the pronunciation content of the first character is to sing a song, then the content of the notification message can be "XX character is singing for you". The terminal device starts the first application based on the user operation.

[0059] In both cases, after the first application is started, the video data is displayed in the first application. For example, after the first application is started, a conversation page for a conversation with the first character is entered, and the video data of the first character is displayed in the conversation page. The video data of the first character can be displayed as a background for the conversation between the user and the first character.

[0060] Figure 2a A schematic diagram of starting a first application program provided by an embodiment of the present disclosure is shown in FIG. Figure 2a As shown, the user can start the first application by triggering a notification message of the first application. Figure 2b A schematic diagram of video data of a first character provided by an embodiment of the present disclosure, such as Figure 2b As shown, after the user starts the first application by triggering the icon of the first application or a notification message, the first application automatically enters a conversation page for a conversation with the first character, and in the conversation page, the video data of the first character is displayed as the conversation background between the user and the first character. In one example, the video data can be played only once as the conversation background. If the user triggers the conversation background, such as long pressing the conversation background, the video data is played repeatedly.

[0061] It can be seen that through this embodiment, the video data of the first character can be generated in advance and displayed after the user enters the first application. Moreover, since the content of the notification message of the first application is related to the pronunciation content of the first character in the video data, the content of the video data can be better represented by the notification content, thereby improving the user's experience of browsing the video data.

[0062] In one embodiment, obtaining video data generated for a first character in a first application includes: If the conversation heat between the first character and the conversation object in the first application meets the heat requirement, providing a creation interface to the conversation object; the creation interface is used to create video data; The video data created by the dialogue object for the first role based on the creation interface is obtained.

[0063] In this embodiment, the dialogue object, such as a user, can customize and generate the first character video data during the dialogue with the first character. Specifically, if the conversation heat between the first character and the dialogue object in the first application meets the heat requirement, a creation interface is provided to the dialogue object, and the creation interface is used to create video data. Among them, the conversation heat can be determined based on the conversation duration and / or the number of conversation rounds between the first character and the dialogue object. If the conversation heat between the first character and the dialogue object in the first application is greater than the heat threshold, a creation interface is provided to the dialogue object, and the creation interface can be a creation page, through which the user creates video data.

[0064] In the creation page, the user can specify the first image of the first character, specify the target dialogue message, and also specify the action requirements of the first character, thereby customizing the video data of the first character.

[0065] It can be seen that through this embodiment, when the conversation between the user and the first character is very heated, a creation interface can also be provided to the user to realize customized creation of the video data of the first character.

[0066] Figure 3 A schematic diagram of a process for generating and displaying video data of a first character provided by an embodiment of the present disclosure, such as Figure 3 As shown, the process includes: Step S302, obtaining a conversation background between a first character and a conversation partner in a first application as a first image; Step S304, determining the message heat of each dialogue message sent by the first character to the dialogue object, determining a first dialogue message from each dialogue message according to the message heat, and generating pronunciation content based on the first dialogue message; Step S306, obtaining a preset action requirement of the first character; Step S308, generating video data of the first character according to the first image, pronunciation content and action requirements through a generative model; Step S310, in response to a user triggering an icon of the first application or a notification message operation, starting the first application; Step S312, controlling the first application to enter a conversation page for conversing with the first character, in which the video data of the first character is displayed as a conversation background between the user and the first character.

[0067] In this embodiment, video data of the first character may be automatically generated in advance and displayed after the user enters the first application.

[0068] Figure 4 A schematic diagram of a process for generating and displaying video data of a first character provided by another embodiment of the present disclosure, such as Figure 4 As shown, the process includes: Step S402, determining that the conversation heat between the first character and the user in the first application meets the heat requirement; Step S404, providing a creation page to the user; Step S406, obtaining the first image of the first character, the target dialogue message and the action requirement of the first character specified by the user based on the creation page, and generating pronunciation content based on the target dialogue message; Step S408, generating video data of the first character according to the first image, pronunciation content and action requirements through a generative model; Step S410: display the video data in the first application.

[0069] In this embodiment, video data of the first character may be generated according to a user's customized operation and displayed in the first application.

[0070] It is worth noting that the process of generating video data in this embodiment can be executed by the server, by the terminal setting, or by the server and the terminal device in cooperation. For example, in one case, after the terminal device in the above embodiments determines that the target dialogue message is obtained, it can also send the target dialogue message to the server, triggering the server to generate pronunciation content based on the target dialogue message, and the terminal device sends the first image to the server, triggering the server to obtain the action requirement through the generative model, and generate the video data of the first character according to the action requirement, the first image and the pronunciation content.

[0071] For example, in another case, after the terminal device in each of the above embodiments determines that the target dialogue message, the first image and the action requirement set by the user are obtained, the target dialogue message, the first image and the action requirement can also be sent to the server, triggering the server to generate the pronunciation content based on the target dialogue message, and triggering the server to generate the video data of the first character according to the action requirement, the first image and the pronunciation content through the generative model. Examples of the terminal device and the server cooperating to generate video data also include various situations based on the technical concept of the present embodiment, which are not given one by one here. Of course, the process of obtaining the first image, the target dialogue message and the action requirement and generating the video data can also be completely executed by the server, and the terminal device is responsible for receiving and displaying the video data sent by the server.

[0072] In one embodiment, after presenting the video data in the first application, the method further includes: In response to a trigger event based on the video data, at least one of the following processes is performed: Download video data, share video data, comment on video data, like video data, and set a conversation background between the first character and the conversation object based on the video data.

[0073] In this embodiment, after displaying the video data in the first application, the user can also trigger the video data. In response to the trigger operation, the terminal system can perform at least one of the above processing, for example, setting the video data or key frames in the video data as the conversation background between the first character and the dialogue object, thereby realizing the interaction between the user and the video data and improving the conversation experience between the user and the first character.

[0074] Based on the contents of the above embodiments, in one case, after acquiring the first image, the terminal device can also drive the first character in the first image to move for a specified time to obtain audio-free video data of the first character. In the video data, the first character only needs to move for a specified time according to the action requirements, and there is no need to output pronunciation content, that is, the video data does not have a corresponding audio output. After the user enters the first application through an icon or notification message, the audio-free video data can be displayed to the user in the first application, and automatically set as the conversation background between the user and the first character. The user can forward, share, like, and comment on the audio-free video data.

[0075] In another case, when the conversation between the user and the first character is heated, a creation interface can be provided to the user, and the audio-free video data can be generated according to the first image and the action requirements and action duration of the first character customized by the user based on the creation interface. The user can forward, share, like, and comment on the audio-free video data, and also set it as the conversation background.

[0076] In summary, through the above embodiments, in a scenario where the conversation object has a conversation with the first character in the first application, the pronunciation content can be determined according to the conversation content between the first character and the conversation object and video data of the first character can be generated. In the video data, the first character performs actions matching the pronunciation content and outputs the pronunciation content, thereby achieving the effect of recording the pronunciation content through the video data, that is, achieving the effect of recording the wonderful conversation between the first character and the conversation object through the video data, thereby improving the efficiency of recording the wonderful conversation.

[0077] Figure 5 A structural diagram of a dialog processing device provided by an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the device comprises: A data acquisition unit 51 is used to acquire video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The data display unit 52 is used to display the video data in the first application.

[0078] Optionally, a generating unit is further included, configured to: Acquire a first image of the first character, and acquire the pronunciation content; generating the video data according to the first image and the pronunciation content; The first character has a first image in the first image; the first character has a second image in the video data; and the second image matches the first image.

[0079] Optionally, the generating unit is specifically used for: Acquire a conversation background between the first character and the conversation object in the first application as the first image; the conversation background is used to represent the first character; or, An image of the first character specified by the dialogue object is acquired as the first image.

[0080] Optionally, the generating unit is specifically used for: Acquire a first dialogue message sent by the first role to the dialogue object as a target dialogue message, or acquire a second dialogue message between the first role specified by the dialogue object and the dialogue object as a target dialogue message; The pronunciation content is generated based on the target dialogue message.

[0081] Optionally, the generating unit is further specifically configured to: Determine the message heat of each conversation message sent by the first role to the conversation object; According to the message popularity, the first dialog message is determined in the dialog messages, and the first dialog message is used as the target dialog message.

[0082] Optionally, the generating unit is further specifically configured to: Using the content in the target dialogue message as the pronunciation content; or, The content associated with the content in the target dialogue message is used as the pronunciation content.

[0083] Optionally, the generating unit is specifically used for: Determine, by means of a generative model, a pronunciation duration corresponding to the pronunciation content, and obtain an action requirement of the first character; Through the generative model, the first character in the first image is rendered according to the action requirements, the pronunciation content and the pronunciation duration to obtain the video data; the rendered first character in the video data moves according to the action requirements, the pronunciation content and the pronunciation duration.

[0084] Optionally, the generating unit is further specifically configured to: Obtaining the action requirement of the first character specified by the dialogue object or set by the system; the action requirement of the first character includes at least one of the action part requirement, action angle requirement, and action direction requirement of the first character.

[0085] Optionally, the data display unit is specifically used for: In response to a trigger event for the first application, starting the first application; the trigger event includes a trigger event for an icon or a notification message of the first application; the content of the notification message is related to the pronunciation content; After the first application is launched, the video data is displayed in the first application.

[0086] Optionally, the data acquisition unit is specifically used to: If the conversation heat between the first character and the conversation object in the first application meets the heat requirement, providing a creation interface to the conversation object; the creation interface is used to create the video data; The video data created by the dialogue object for the first role based on the creation interface is obtained.

[0087] Optionally, an operating unit is further included, for: After the video data is displayed in the first application, in response to a trigger event based on the video data, at least one of the following processes is performed: Download the video data, share the video data, comment on the video data, like the video data, and set a conversation background between the first character and the conversation object based on the video data.

[0088] The dialogue processing device in the embodiment of the present disclosure can implement each process of the above-mentioned dialogue processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0089] An embodiment of the present disclosure further provides an electronic device, Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 6 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 601 and memory 602, and one or more applications or data may be stored in the memory 602. Among them, the memory 602 may be a temporary storage or a permanent storage. The application stored in the memory 602 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 601 may be configured to communicate with the memory 602 to execute a series of computer executable instructions in the memory 602 on the electronic device. The electronic device may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input or output interfaces 605, one or more keyboards 606, etc.

[0090] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process: Obtain video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The video data is presented in the first application.

[0091] The electronic device in the embodiment of the present disclosure can implement each process of the above-mentioned dialogue processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0092] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Obtain video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The video data is presented in the first application.

[0093] The computer-readable storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned dialogue processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0094] Another embodiment of the present disclosure further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following process is implemented: Obtain video data generated for a first character in a first application; the first character is created based on a generative model; the first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The video data is presented in the first application.

[0095] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned dialogue processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0096] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0097] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0098] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0099] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0100] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0101] It should be understood by those skilled in the art that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0106] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0107] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0108] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A method for processing a conversation, characterized in that: include: Obtain video data generated for a first character in a first application; The first role is created based on a generative model; The first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; The video data is presented in the first application.

2. The method according to claim 1, characterized in that Also includes: Acquire a first image of the first character, and acquire the pronunciation content; generating the video data according to the first image and the pronunciation content; The first character has a first image in the first image; the first character has a second image in the video data; and the second image matches the first image.

3. The method according to claim 2, characterized in that The obtaining of the first image of the first character comprises: Acquire a conversation background between the first character and the conversation object in the first application as the first image; the conversation background is used to represent the first character; or, An image of the first character specified by the dialogue object is acquired as the first image.

4. The method according to claim 2, characterized in that: The obtaining of the pronunciation content includes: Acquire a first dialogue message sent by the first role to the dialogue object as a target dialogue message, or acquire a second dialogue message between the first role specified by the dialogue object and the dialogue object as a target dialogue message; The pronunciation content is generated based on the target dialogue message.

5. The method according to claim 4, characterized in that The obtaining, as the target dialogue message, the first dialogue message sent by the first role to the dialogue object includes: Determine the message heat of each conversation message sent by the first role to the conversation object; According to the message popularity, the first dialog message is determined in the dialog messages, and the first dialog message is used as the target dialog message.

6. The method according to claim 4, characterized in that The generating the pronunciation content based on the target dialogue message includes: Using the content in the target dialogue message as the pronunciation content; or, The content associated with the content in the target dialogue message is used as the pronunciation content.

7. The method according to claim 2, characterized in that The step of generating the video data according to the first image and the pronunciation content includes: Determine, by means of a generative model, a pronunciation duration corresponding to the pronunciation content, and obtain an action requirement of the first character; Through the generative model, the first character in the first image is rendered according to the action requirements, the pronunciation content and the pronunciation duration to obtain the video data; the rendered first character in the video data moves according to the action requirements, the pronunciation content and the pronunciation duration.

8. The method according to claim 7, characterized in that The obtaining the action requirement of the first role includes: Obtaining the action requirement of the first character specified by the dialogue object or set by the system; the action requirement of the first character includes at least one of the action part requirement, action angle requirement, and action direction requirement of the first character.

9. The method according to claim 1, characterized in that: The displaying of the video data in the first application comprises: In response to a trigger event for the first application, starting the first application; the trigger event includes a trigger event for an icon or a notification message of the first application; the content of the notification message is related to the pronunciation content; After the first application is launched, the video data is displayed in the first application.

10. The method according to claim 1, characterized in that The acquiring of video data generated for the first character in the first application comprises: If the conversation heat between the first character and the conversation object in the first application meets the heat requirement, providing a creation interface to the conversation object; the creation interface is used to create the video data; The video data created by the dialogue object for the first role based on the creation interface is obtained.

11. The method according to claim 1, characterized in that: After displaying the video data in the first application, the method further includes: In response to a trigger event based on the video data, at least one of the following processes is performed: Download the video data, share the video data, comment on the video data, like the video data, and set a conversation background between the first character and the conversation object based on the video data.

12. A dialogue processing device, characterized in that: include: A data acquisition unit, configured to acquire video data generated for a first character in a first application; The first role is created based on a generative model; The first character in the first application is used to have a conversation with a dialogue partner; the first character in the video data performs an action matching a pronunciation content and outputs the pronunciation content; the pronunciation content is related to a conversation content, and the conversation content is a conversation content between the first character and the dialogue partner; A data display unit is used to display the video data in the first application.

13. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images