Resource display method and device, terminal and storage medium

By configuring voice text and voice timbre for emoticons, the combination of vision and auditory is achieved, and the problem of single emotional expression dimensions of emoticons is solved, which enhances user interaction and communication effects.

CN118921532APending Publication Date: 2024-11-08BEIJING DAJIA INTERNET INFORMATION TECH CO LTD

Patent Information

Application Number
CN202411337829.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-24
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The emotional expression dimensions of existing emoticons are relatively single and rely only on vision, resulting in a single user interaction effect and a lack of rich auditory experience.

Method used

By obtaining the voice text and voice timbre configured by the user for the initial media resource in the resource editing interface, a target media resource combining voice and media resource is generated, enabling it to express emotions from both visual and auditory dimensions.

Benefits of technology

It enriches the emotional expression dimensions of emoticons, enhances the interaction between users, and improves the fun and interactive effects of communication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118921532A_ABST
    Figure CN118921532A_ABST
Patent Text Reader

Abstract

The invention provides a resource display method and device, a terminal and a storage medium, and belongs to the technical field of computers. According to the method, the voice text and the voice timbre configured by the user for the initial media resource on the resource editing interface are acquired, so that the terminal can generate the target media resource combining the voice resource and the media resource based on the voice text, the voice timbre and the initial media resource; even if the target media resource can jointly express the emotion of the user from the visual dimension and the auditory dimension, the emotion expression dimension of the target media resource is enriched, the interaction effect of the user using the target media resource for communication is enriched, and the interactivity between the users is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a resource display method, device, terminal and storage medium. Background Art

[0002] With the development of Internet technology, the use of emoticons for communication in online social networking has become increasingly popular. Among them, emoticons are a way of expressing emotions using images, which can make up for the weakness of boring text communication and increase the fun of interactive communication. However, current emoticons can only express users' emotions from a visual dimension, and the emotional expression dimension of emoticons is relatively single, resulting in a relatively single interactive effect when users communicate through emoticons. Therefore, how to enrich the emotional expression dimension of emoticons to enhance the interactivity between users is a technical problem that needs to be solved. Summary of the invention

[0003] The present disclosure provides a resource display method, device, terminal and storage medium, which can enable the target media resource to express the user's emotions from the visual dimension and the auditory dimension, thereby enriching the emotional expression dimension of the target media resource, and further enriching the interactive effect of the user using the target media resource to communicate, and enhancing the interactivity between users. The technical solution of the present disclosure is as follows.

[0004] According to one aspect of an embodiment of the present disclosure, a resource display method is provided, including:

[0005] In response to a trigger operation on a resource creation entry, displaying a resource editing interface, wherein the resource editing interface displays an initial media resource;

[0006] In response to the voice configuration operation on the initial media resource, acquiring the voice text and voice timbre configured for the initial media resource;

[0007] Based on the speech text, the speech timbre and the initial media resource, a target media resource is generated, wherein the target media resource includes a target speech and the initial media resource, the content of the target speech is the speech text, and the timbre of the target speech is the speech timbre;

[0008] The target media resource is displayed.

[0009] According to another aspect of an embodiment of the present disclosure, there is provided a resource display device, including:

[0010] A first display unit is configured to display a resource editing interface in response to a trigger operation on a resource creation portal, wherein the resource editing interface displays an initial media resource;

[0011] A first acquisition unit is configured to acquire a voice text and a voice timbre configured for the initial media resource in response to a voice configuration operation on the initial media resource;

[0012] A generating unit is configured to generate a target media resource based on the voice text, the voice timbre and the initial media resource, wherein the target media resource includes a target voice and the initial media resource, the content of the target voice is the voice text, and the timbre of the target voice is the voice timbre;

[0013] The first display unit is further configured to display the target media resource.

[0014] In some embodiments, the first display unit is configured to display multiple candidate media resources in response to a trigger operation on a resource creation portal; in response to a selection operation on any candidate media resource, display the initial media resource on the resource editing interface based on the selected candidate media resource.

[0015] In some embodiments, the first display unit is configured to display the selected candidate media resource in the resource editing interface when the selected candidate media resource is an image resource; or to edit the selected candidate media resource to obtain a media resource including at least one video frame when the selected candidate media resource is a video resource; and to display the media resource including at least one video frame in the resource editing interface.

[0016] In some embodiments, the first acquisition unit is configured to acquire input voice text in response to an input operation in the resource editing interface; or to perform image recognition on the initial media resource to obtain the voice text.

[0017] In some embodiments, the first acquisition unit is configured to perform image recognition on the initial media resource to obtain an image recognition result of the initial media resource, wherein the image recognition result includes at least one of text information, object information, and emotion information, wherein the text information is used to indicate the image text in the initial media resource, the object information is used to indicate the object information of the object in the initial media resource, and the emotion information is used to indicate the emotion represented by the object in the initial media resource; based on the image recognition result, the speech text is generated.

[0018] In some embodiments, the first acquisition unit is configured to generate the voice text based on the text information when the initial media resource contains image text; or to generate the voice text based on at least one of the object information and the emotion information when the initial media resource does not contain image text.

[0019] In some embodiments, the resource editing interface displays at least one candidate voice timbre and image information corresponding to each candidate voice timbre, the image information including a virtual image corresponding to the candidate voice timbre and a name of the virtual image;

[0020] The first acquisition unit is configured to acquire the selected candidate voice timbre in response to a selection operation on any candidate voice timbre.

[0021] In some embodiments, after acquiring the voice text, the device further includes:

[0022] A screening unit, based on the initial media resource and the voice text, screening a plurality of candidate voice timbres to obtain at least one candidate voice timbre;

[0023] The first display unit is configured to display the at least one candidate voice timbre on the resource editing interface;

[0024] The first acquisition unit is configured to acquire the selected candidate voice timbre in response to a selection operation on any candidate voice timbre.

[0025] In some embodiments, when the initial media resource includes at least one object, the speech text includes the speech text of the at least one object;

[0026] The screening unit is configured to perform image recognition on the initial media resource to obtain object information of at least one object in the initial media resource; determine the dialogue role corresponding to each object based on the object information of the at least one object and the voice text of the at least one object; for any object, based on the dialogue role corresponding to the object, screen the multiple candidate voice timbres to obtain at least one candidate voice timbre matching the object.

[0027] In some embodiments, the screening unit is configured to determine the object relationship of the at least one object based on the object information of the at least one object and the voice text of the at least one object; for any object, determine the dialogue role corresponding to the object based on the object information of the object and the object relationship.

[0028] In some embodiments, the apparatus further comprises:

[0029] a determination unit configured to recognize, for any object, a voice text of the object and determine a text subject of the voice text;

[0030] The screening unit is configured to screen the multiple candidate voice timbres based on the dialogue role corresponding to the object and the text subject matter of the voice text.

[0031] In some embodiments, the at least one candidate voice timbre includes a voice timbre of a first object, the first object being a user object currently logged in by the terminal;

[0032] The device also includes:

[0033] A second display unit is configured to display a timbre input interface in response to a timbre adding operation on the first object in the resource editing interface;

[0034] The second acquisition unit is configured to acquire the voice signal input by the first object in response to the voice input operation in the timbre recording interface; and acquire the voice timbre of the first object based on the voice signal.

[0035] In some embodiments, the at least one candidate voice timbre includes a voice timbre of a second object, the second object is an object associated with the first object, and the first object is a user object currently logged in by the terminal;

[0036] The device also includes:

[0037] a sending unit, configured to send a timbre entry link to the second object in response to a timbre adding operation on the second object in the resource editing interface;

[0038] The second acquisition unit is further configured to acquire a voice signal input by the second object in a timbre recording interface corresponding to the timbre recording link when the second object successfully records voice content; and acquire the voice timbre of the second object based on the voice signal.

[0039] In some embodiments, the apparatus further comprises:

[0040] A third display unit is configured to display the target media resource and a voice playback control of the target media resource on the conversation interface in response to a sending operation on the target media resource in the conversation interface;

[0041] The playing unit is configured to play the target voice in the target media resource in response to a triggering operation on the voice playing control.

[0042] According to another aspect of an embodiment of the present disclosure, a terminal is provided, the terminal including:

[0043] one or more processors;

[0044] A memory for storing program codes executable by the processor;

[0045] The processor is configured to execute the program code to implement the resource display method.

[0046] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When a program code in the computer-readable storage medium is executed by a processor of a terminal, the terminal is enabled to execute the resource display method.

[0047] According to another aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the resource display method is implemented.

[0048] The disclosed embodiment provides a resource display method, which obtains the voice text and voice timbre configured by the user for the initial media resource in the resource editing interface, so that the terminal can generate a target media resource that combines the voice resource and the media resource based on the voice text, the voice timbre and the initial media resource, that is, the target media resource can express the user's emotions from both the visual dimension and the auditory dimension, thereby enriching the emotional expression dimension of the target media resource, and further enriching the interactive effect of the user using the target media resource to communicate, thereby enhancing the interactivity between users.

[0049] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0051] Figure 1 is a schematic diagram of an implementation environment of a resource display method according to an exemplary embodiment;

[0052] Figure 2 is a flow chart of a resource display method according to an exemplary embodiment;

[0053] Figure 3 is a flow chart showing another resource display method according to an exemplary embodiment;

[0054] Figure 4 is a schematic diagram of a resource creation portal according to an exemplary embodiment;

[0055] Figure 5 is a schematic diagram of a resource editing interface according to an exemplary embodiment;

[0056] Figure 6 is a schematic diagram of a timbre recording process according to an exemplary embodiment;

[0057] Figure 7 is a schematic diagram of another timbre recording process according to an exemplary embodiment;

[0058] Figure 8 is a schematic diagram of a conversation interface according to an exemplary embodiment;

[0059] Fig. 9 is a block diagram of a resource display device according to an exemplary embodiment;

[0060] Fig.10 is a block diagram of another resource display device according to an exemplary embodiment;

[0061] Fig.11 It is a block diagram of a terminal according to an exemplary embodiment. DETAILED DESCRIPTION

[0062] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0063] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0064] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the initial image resources, voice text, voice timbre, etc. involved in this disclosure are all obtained with full authorization.

[0065] Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a resource display method according to an exemplary embodiment. Figure 1The implementation environment specifically includes: a terminal 101 and a server 102. The terminal 101 can be connected to the server 102 via a wireless network or a wired network.

[0066] Terminal 101 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, and a laptop computer. A target application is installed or running on terminal 101. The target application is any type of application that can provide instant messaging functions, such as video applications, information applications, or social applications. Users can use terminal 101 to send media resources in the target application to other users who use the target application. Media resources include text, voice, video, images, emoticons, etc. Among them, the target application is associated with server 102, and server 102 provides background services.

[0067] The terminal 101 may generally refer to one of a plurality of terminals. This embodiment is described by taking the terminal 101 as an example. Those skilled in the art may know that the number of the above terminals may be more or less. For example, the above terminals may be several, or the above terminals may be dozens or hundreds, or a greater number. The embodiment of the present disclosure does not limit the number and device type of the terminals.

[0068] The server 102 is at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Optionally, the number of the above servers may be more or less, which is not limited in the embodiments of the present disclosure. Of course, the server 102 may also include other functional servers to provide more comprehensive and diversified services. In some embodiments, the server 102 undertakes the main computing work, and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work, and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 use a distributed computing architecture for collaborative computing. Optionally, the number of the above servers may be more or less, which is not limited in the embodiments of the present disclosure.

[0069] Figure 2 is a flow chart of a resource display method according to an exemplary embodiment. Figure 2 As shown, the method is executed by the terminal and includes the following steps.

[0070] In step 201, in response to a triggering operation on a resource creation portal, the terminal displays a resource editing interface, and the resource editing interface displays an initial media resource.

[0071] In an embodiment of the present disclosure, a terminal is installed or running an application that supports instant messaging. A user can send media resources to other users who use the application in a conversation interface provided by the application. The media resources can be image resources or video resources, and the image resources can be static images or dynamic images. The conversation interface provides an input function for media resources. The terminal can display multiple media resources to be sent and a resource creation portal for creating media resources in the conversation interface. Accordingly, in response to a trigger operation on the resource creation portal, the terminal can display a resource editing interface. Among them, the resource editing interface displays the initial media resources uploaded by the user. The resource editing interface is used to edit the initial media resources to generate new media resources.

[0072] In step 202, in response to the voice configuration operation on the initial media resource, the terminal obtains the voice text and voice timbre configured for the initial media resource.

[0073] In the embodiment of the present disclosure, the voice configuration operation refers to the dubbing operation of the initial media resource. Accordingly, in response to the voice configuration operation of the initial media resource, the terminal can obtain the voice text and voice timbre configured by the user for the initial media resource. Among them, the voice text refers to the content of the voice, and the voice timbre refers to the pronunciation characteristics of the voice, such as gentle lady voice, sweet girl voice, mature uncle voice, cheerful youth voice, etc.

[0074] In step 203, the terminal generates a target media resource based on the voice text, the voice timbre and the initial media resource, the target media resource includes a target voice and the initial media resource, the content of the target voice is the voice text, and the timbre of the target voice is the voice timbre.

[0075] In the disclosed embodiment, after acquiring the voice text and voice timbre, the terminal can generate the narration of the initial media resource or the dubbing of the object in the initial media resource, that is, the target voice, based on the voice text and voice timbre. Further, the terminal can generate the target media resource based on the target voice and the initial media resource. The object in the initial media resource can be a person or an animal.

[0076] In step 204, the terminal displays the target media resource.

[0077] In the embodiment of the present disclosure, after obtaining the target media resource, the terminal can cancel displaying the resource editing interface and display the target media resource in the conversation interface so that the user can use the target media resource to interact with other users.

[0078] The disclosed embodiment provides a resource display method, which obtains the voice text and voice timbre configured by the user for the initial media resource in the resource editing interface, so that the terminal can generate a target media resource that combines the voice resource and the media resource based on the voice text, the voice timbre and the initial media resource, that is, the target media resource can express the user's emotions from both the visual dimension and the auditory dimension, thereby enriching the emotional expression dimension of the target media resource, and further enriching the interactive effect of the user using the target media resource to communicate, thereby enhancing the interactivity between users.

[0079] In some embodiments, in response to a trigger operation on a resource creation portal, a resource editing interface is displayed, including:

[0080] In response to a triggering operation on a resource creation entry, displaying a plurality of candidate media resources;

[0081] In response to a selection operation on any candidate media resource, an initial media resource is displayed on the resource editing interface based on the selected candidate media resource.

[0082] In the disclosed embodiment, by displaying a plurality of candidate media resources, it is possible for a user to quickly select an initial media resource to be uploaded, which not only meets the user's personalized needs for initial media resources, but also improves the efficiency of human-computer interaction.

[0083] In some embodiments, based on the selected candidate media resource, displaying the initial media resource on the resource editing interface includes:

[0084] When the selected candidate media resource is an image resource, the selected candidate media resource is displayed on the resource editing interface; or,

[0085] When the selected candidate media resource is a video resource, the selected candidate media resource is edited to obtain a media resource including at least one video frame;

[0086] The media resource containing at least one video frame is displayed in the resource editing interface.

[0087] In the disclosed embodiment, the selected candidate media resources are automatically processed based on the resource type of the selected candidate media resources to obtain the initial media resources, so that the user does not need to manually edit the selected candidate media resources to obtain the initial media resources displayed on the resource editing interface, thereby improving the efficiency of human-computer interaction.

[0088] In some embodiments, the process of acquiring the voice text includes:

[0089] In response to an input operation in the resource editing interface, obtaining input voice text; or,

[0090] Perform image recognition on the initial media resources to obtain speech text.

[0091] In the disclosed embodiment, by obtaining the voice text manually input by the user as the voice text of the object in the initial media resource, the user can freely set the voice text of the object in the initial media resource, thereby satisfying the user's personalized needs for voice text. By obtaining the voice text generated based on the initial media resource as the voice text of the object in the initial media resource, the user does not need to manually configure the voice text of the initial media resource, thereby improving the efficiency of obtaining the voice text, thereby improving the efficiency of generating the target media resource, and improving the efficiency of human-computer interaction.

[0092] In some embodiments, performing image recognition on the initial media resource to obtain the voice text includes:

[0093] Performing image recognition on the initial media resource to obtain an image recognition result of the initial media resource, wherein the image recognition result includes at least one of text information, object information, and emotion information, wherein the text information is used to indicate image text in the initial media resource, the object information is used to indicate object information of an object in the initial media resource, and the emotion information is used to indicate an emotion represented by the object in the initial media resource;

[0094] Generate speech text based on the image recognition results.

[0095] In the embodiments of the present disclosure, by automatically generating voice text based on at least one of the text information, object information and emotion information of the initial media resource, not only can the user avoid manually configuring the voice text of the initial media resource, thereby improving the efficiency of human-computer interaction, but also the accuracy of the voice text can be improved and the generation quality of the target media resource can be improved.

[0096] In some embodiments, generating speech text based on the image recognition result includes:

[0097] In the case where the initial media resource includes image text, generating speech text based on the text information; or,

[0098] In the case that the initial media resource does not include the image text, the speech text is generated based on at least one of the object information and the emotion information.

[0099] In the disclosed embodiment, by determining the image text contained in the initial media resource as the voice text of the object in the initial media resource, the voice text can be quickly and accurately obtained while improving the efficiency of human-computer interaction. By generating voice text based on the object information and emotional information of the object in the initial media resource when the initial media resource does not include the image text, the emotional information expressed by the voice text can be enriched while improving the efficiency of human-computer interaction, making the voice generated based on the voice text more vivid and vivid, thereby enriching the emotional expression effect of the target media resource.

[0100] In some embodiments, the resource editing interface displays at least one candidate voice timbre and image information corresponding to each candidate voice timbre, and the image information includes a virtual image corresponding to the candidate voice timbre and a name of the virtual image;

[0101] The process of acquiring speech timbre includes:

[0102] In response to a selection operation on any candidate voice timbre, the selected candidate voice timbre is acquired.

[0103] In the disclosed embodiment, by displaying at least one candidate voice timbre in the resource editing interface, the user can quickly select the voice timbre of the object in the initial media resource, which not only meets the user's personalized needs for voice timbre, but also improves the efficiency of human-computer interaction.

[0104] In some embodiments, after obtaining the voice text, the method further includes:

[0105] Based on the initial media resources and the voice text, a plurality of candidate voice timbres are screened to obtain at least one candidate voice timbre;

[0106] Display at least one candidate voice timbre in the resource editing interface;

[0107] The process of acquiring speech timbre includes:

[0108] In response to a selection operation on any candidate voice timbre, the selected candidate voice timbre is acquired.

[0109] In the embodiments of the present disclosure, at least one candidate voice timbre suitable for the object is recommended to the user based on the initial media resource and the voice text of the object in the initial media resource, so that the user does not need to manually search for the voice timbre suitable for the object, thereby improving the efficiency of obtaining the voice timbre and improving the efficiency of human-computer interaction.

[0110] In some embodiments, where the initial media resource includes at least one object, the spoken text includes the spoken text of the at least one object;

[0111] Based on the initial media resources and the voice text, multiple candidate voice timbres are screened to obtain at least one candidate voice timbre, including:

[0112] Performing image recognition on the initial media resource to obtain object information of at least one object in the initial media resource;

[0113] Determine a dialogue role corresponding to each object based on object information of at least one object and a voice text of at least one object;

[0114] For any object, multiple candidate voice timbres are screened based on the dialogue role corresponding to the object to obtain at least one candidate voice timbre matching the object.

[0115] In the embodiments of the present disclosure, by recommending candidate voice timbre suitable for the object in the initial media resource according to the dialogue role played by the object in the dialogue process, the accuracy of at least one candidate voice timbre recommended by the terminal to the user can be improved while improving the efficiency of human-computer interaction, so that the user can quickly select a voice timbre suitable for the object in the initial media resource.

[0116] In some embodiments, based on the object information of at least one object and the voice text of at least one object, determining the dialogue role corresponding to each object includes:

[0117] determining an object relationship of at least one object based on object information of at least one object and a speech text of at least one object;

[0118] For any object, the dialogue role corresponding to the object is determined based on the object information and object relationship of the object.

[0119] In the embodiments of the present disclosure, by determining the dialogue role played by the object based on the object relationship between the objects in the initial media resources, the accuracy of the dialogue role determined by the terminal can be improved, and the accuracy of the candidate voice timbre suitable for the dialogue role determined by the terminal can be improved, thereby improving the efficiency of human-computer interaction while allowing the user to quickly select a voice timbre suitable for the object in the initial media resources.

[0120] In some embodiments, the method further comprises:

[0121] For any object, recognize the speech text of the object and determine the text subject of the speech text;

[0122] Based on the dialogue role corresponding to the object, multiple candidate voice timbres are screened, including:

[0123] Based on the dialogue role corresponding to the object and the text subject matter of the voice text, a variety of candidate voice timbres are screened.

[0124] In the embodiments of the present disclosure, by recommending candidate voice timbre suitable for the object in the initial media resource based on the dialogue role played by the object during the dialogue process and the text subject matter of the object's voice text, it is possible to further improve the accuracy of at least one candidate voice timbre recommended by the terminal to the user while improving the efficiency of human-computer interaction, so that the user can quickly select a voice timbre suitable for the object in the initial media resource.

[0125] In some embodiments, at least one candidate voice timbre includes a voice timbre of a first object, the first object being a user object currently logged in by the terminal;

[0126] The method also includes:

[0127] In response to a timbre adding operation on the first object in the resource editing interface, displaying a timbre recording interface;

[0128] In response to a voice input operation in the timbre recording interface, acquiring a voice signal input by a first subject;

[0129] Based on the speech signal, a speech timbre of the first object is acquired.

[0130] In the embodiments of the present disclosure, by recording the user's own voice timbre, not only can the user select his own voice timbre as the voice timbre of the object in the initial media resource when the object in the initial media resource is the user himself, thereby improving the adaptability of the object and the voice timbre, but the user can also obtain a voice with the user's voice characteristics without manually dubbing the object, thereby reducing the operating cost of dubbing the object in the initial media resource and improving the efficiency of human-computer interaction.

[0131] In some embodiments, at least one candidate voice timbre includes a voice timbre of a second object, the second object is an object associated with the first object, and the first object is a user object currently logged in by the terminal;

[0132] The method also includes:

[0133] In response to a timbre adding operation on the second object in the resource editing interface, sending a timbre recording link to the second object;

[0134] When the second object successfully records the voice content, obtaining the voice signal input by the second object in the timbre recording interface corresponding to the timbre recording link;

[0135] Based on the voice signal, a voice timbre of the second object is acquired.

[0136] In the embodiments of the present disclosure, by recording the voice timbre of other users who are associated with the user, not only can the user select the friend's voice timbre as the voice timbre of the object in the initial media resource when the object in the initial media resource is the user's friend, thereby improving the adaptability of the object and the voice timbre, but the user can also obtain a voice with the characteristics of the friend's voice without inviting the friend to manually dub the object, thereby reducing the operating cost of dubbing the object in the initial media resource and improving the efficiency of human-computer interaction.

[0137] In some embodiments, the method further comprises:

[0138] In response to a sending operation on a target media resource in the conversation interface, displaying the target media resource and a voice playback control of the target media resource in the conversation interface;

[0139] In response to a triggering operation on a voice playing control, a target voice in a target media resource is played.

[0140] In the disclosed embodiment, the terminal can play the target voice in the target media resource in the conversation interface, so that the user can feel the information expressed by the target media resource from a visual dimension, and can also feel the information expressed by the target media resource from an auditory dimension, thereby enriching the interactive effect of the user using the target media resource to communicate and enhancing the interactivity between users.

[0141] Above Figure 2 The figure shows a process of a resource display method of the present disclosure. The resource display scheme provided by the present disclosure is further described below. Figure 3 is a flowchart of another resource display method according to an exemplary embodiment, the method is executed by a terminal, see Figure 3 , the method comprises the following steps.

[0142] In step 301, in response to a triggering operation on a resource creation portal, the terminal displays a plurality of candidate media resources.

[0143] In an embodiment of the present disclosure, the terminal can determine the initial media resource displayed in the resource editing interface based on the user's selection operation of any candidate media resource among multiple candidate media resources. Accordingly, in response to the triggering operation of the resource creation portal, the terminal can display multiple candidate media resources for the user to select. Among them, the terminal can set the default jump address after the resource creation portal is triggered. The default jump address can be the address of a local folder for storing media resources shot by the user or downloaded from the network, or the address of a cloud folder for storing application data of an application in the terminal. In the case where the default jump address is the address of a local folder, the multiple candidate media resources displayed by the terminal are media resources stored in the local folder. In the case where the default jump address is the address of a cloud folder corresponding to the application, the multiple candidate media resources displayed by the terminal are media resources stored in the cloud folder, such as image works or video works posted by the user in the application.

[0144] For example, Figure 4 FIG. 1 is a schematic diagram showing a resource creation portal according to an exemplary embodiment. Figure 4 As shown in (a) of FIG. 4 , a plurality of media resources 402 to be sent and a resource creation entry 403 for creating media resources are displayed in the session interface 401. In response to the triggering operation on the resource creation entry 403, the terminal can display Figure 4 The resource upload page 404 shown in (b) of FIG. The resource upload page 404 displays multiple candidate media resources.

[0145] In step 302, in response to a selection operation on any candidate media resource, the terminal displays an initial media resource on a resource editing interface based on the selected candidate media resource.

[0146] In the embodiments of the present disclosure, see Figure 4 In the case where any candidate media resource in the resource upload page 404 is selected, the terminal can respond to the selection operation of the candidate media resource, determine the initial media resource based on the resource type of the selected candidate media resource, and then display the initial media resource in the resource editing interface. By displaying multiple candidate media resources, it is convenient for users to quickly select the initial media resource to be uploaded, which not only meets the user's personalized needs for the initial media resource, but also improves the efficiency of human-computer interaction.

[0147] In some embodiments, the selected candidate media resource may be an image resource or a video resource. Accordingly, the process of the terminal determining the initial media resource based on the resource type of the selected candidate media resource includes the following implementation mode 1 and implementation mode 2.

[0148] Implementation method 1: When the selected candidate media resource is an image resource, the terminal does not need to process the selected candidate media resource, that is, the terminal can directly determine the selected candidate media resource as the initial media resource. Accordingly, the terminal can display the selected candidate media resource on the resource editing interface.

[0149] Implementation method 2: When the selected candidate media resource is a video resource, since most video resources have a large amount of data due to their long duration, in order to improve the editing efficiency of the initial media resource and speed up the generation of the target media resource, the terminal can edit the selected candidate media resource to obtain a media resource containing at least one video frame, that is, the terminal can intercept a media resource with a duration not greater than a preset duration threshold from the candidate media resource, and determine the media resource as the initial media resource. Correspondingly, the terminal can display the media resource containing at least one video frame on the resource editing interface. Among them, the preset duration threshold can be 3 seconds, 5 seconds, 10 seconds, etc., and the embodiment of the present disclosure does not limit this. By automatically processing the selected candidate media resource based on the resource type of the selected candidate media resource to obtain the initial media resource, the user does not need to manually edit the selected candidate media resource to obtain the initial media resource displayed on the resource editing interface, thereby improving the efficiency of human-computer interaction.

[0150] It should be noted that, in response to the voice configuration operation on the initial media resource, the terminal can obtain the voice text configured for the initial media resource by obtaining the voice text input by the user, or by obtaining the voice text generated based on the initial media resource. Accordingly, in the case where the voice text is obtained by obtaining the voice text generated based on the initial media resource, the terminal can perform the following step 303; in the case where the voice text is obtained by obtaining the voice text input by the user, the terminal can perform the following step 304.

[0151] In step 303, the terminal performs image recognition on the initial media resource to obtain voice text.

[0152] In the disclosed embodiment, the terminal can identify the initial media resource displayed on the resource editing interface through an image recognition model to obtain voice text. Among them, the image recognition model can extract features of the input initial media resource to obtain multiple resource features of the initial media resource, and then generate voice text for representing the initial media resource based on the multiple resource features. The image recognition model can be a graph-to-text model based on image processing. By obtaining the voice text generated based on the initial media resource as the voice text of the initial media resource, the user does not need to manually configure the voice text of the initial media resource, which improves the efficiency of obtaining the voice text, thereby improving the efficiency of generating the target media resource and improving the efficiency of human-computer interaction.

[0153] For example, Figure 5 FIG. 1 is a schematic diagram of a resource editing interface according to an exemplary embodiment. Figure 5 As shown in (a) in FIG. 5 , an input box is displayed in the resource editing interface 501. The terminal can display the recognized voice text in the input box for the user to view. The user can also modify the voice text displayed in the input box to improve the accuracy of the voice text.

[0154] In some embodiments, the image recognition model has functions such as text recognition, object recognition, and emotion recognition. Accordingly, the terminal performs image recognition on the initial media resource and can obtain the image recognition result of the initial media resource. Among them, the image recognition result includes at least one of text information, object information, and emotion information. The text information is used to indicate the image text in the initial media resource, the object information is used to indicate the object information of the object in the initial media resource, and the emotion information is used to indicate the emotion represented by the object in the initial media resource. Then, the terminal can generate voice text based on the image recognition result. Among them, the image text is the text displayed in the initial media resource. The object information is the object feature of the object in the initial media resource. Taking the object as a person as an example, the object feature may include the height, body shape, skin color, hairstyle, figure, face and other features of the object. The emotions represented by the emotion information may include emotions such as happiness, anger, sadness, surprise, etc. By automatically generating voice text based on at least one of the text information, object information, and emotion information of the initial media resource, not only can the user not need to manually configure the voice text of the initial media resource, improve the efficiency of human-computer interaction, but also improve the accuracy of the voice text and improve the generation quality of the target media resource.

[0155] In some embodiments, during the image recognition process, the terminal can perform emotion recognition on the text information of the initial media resource to obtain the emotion information of the initial media resource. For example, when the text information is "hahaha", the emotion information of the initial media resource is "happy". When the text information is "I am angry", the emotion information of the initial media resource is "angry".

[0156] In some embodiments, during the image recognition process, the terminal can perform emotion recognition on the object information of the initial media resource to obtain the emotion information of the initial media resource. For example, when the facial features of the object in the object information are raised eyebrows and wide eyes, the emotion information of the initial media resource is "surprise". When the facial features of the object in the object information are downturned corners of the mouth and frowning, the emotion information of the initial media resource is "sadness".

[0157] In some embodiments, the process of generating voice text by the terminal based on the image recognition result includes the following implementation method 1 and implementation method 2.

[0158] Implementation method one: The image text is the text included in the screen of the initial media resource. When the initial media resource contains image text, it indicates that the image recognition result of the initial media resource includes the text information of the initial media resource. Therefore, the terminal can generate voice text based on the text information, that is, the terminal can perform character recognition on the text information of the initial media resource to obtain the text included in the screen of the initial media resource, and then determine the text as voice text. By performing character recognition on the text information of the initial media resource to obtain text and determining it as the voice text of the initial media resource, the voice text can be obtained quickly and accurately while improving the efficiency of human-computer interaction.

[0159] Implementation method 2: When the initial media resource does not contain image text, it indicates that the image recognition result of the initial media resource does not include the text information of the initial media resource, so the terminal can generate voice text based on at least one of the object information and emotional information in the image recognition result. Among them, the voice text is a text used to represent at least one of the object information and emotional information of the initial media resource. For example, when the object information indicates that the object in the initial media resource is a female, and the emotional information indicates that the emotion of the object is sad, the voice content generated by the terminal can be a text reflecting the sad crying of a female, such as "wooooo", "oooooooo", etc. By generating voice text based on the object information and emotional information of the object in the initial media resource when the initial media resource does not include image text, it is possible to improve the efficiency of human-computer interaction while enriching the emotional information expressed by the voice text, making the voice generated based on the voice text more vivid and vivid, thereby enriching the emotional expression effect of the target media resource.

[0160] In some embodiments, the terminal can also send the initial media resource to the server, and the server recognizes the initial media resource through an image recognition model to obtain a voice text. The terminal can receive the voice text returned by the server.

[0161] In step 304, in response to an input operation in the resource editing interface, the terminal obtains the input voice text.

[0162] In the embodiments of the present disclosure, Figure 5 As shown in (a) in FIG. 5 , the input box in the resource editing interface 501 is used not only to display the voice text generated by the terminal based on the initial media resource, but also to input the voice text. Accordingly, the user can input the voice text configured for the initial media resource in the resource editing interface 501 through the input box. By obtaining the voice text manually input by the user as the voice text of the initial media resource, the user can freely set the voice text of the initial media resource, thereby satisfying the user's personalized needs for the voice text.

[0163] In step 305, when at least one candidate voice timbre is displayed on the resource editing interface, in response to a selection operation of any candidate voice timbre, the terminal obtains the voice timbre configured for the initial media resource, and the voice timbre is the selected candidate voice timbre.

[0164] In the embodiment of the present disclosure, the above method is an exemplary method for obtaining the voice timbre in the resource editing interface. Figure 5 As shown in (a), a variety of candidate voice timbres are displayed in the resource editing interface 501. In response to the selection operation of any candidate voice timbre (such as timbre 1), the terminal can obtain the selected candidate voice timbre, that is, the selected candidate voice timbre is used as the voice timbre of the object in the initial media resource 502. The resource editing interface also displays the image information corresponding to each candidate voice timbre, and the image information includes the virtual image corresponding to the candidate voice timbre and the name of the virtual image. For example, taking the candidate voice timbre as a gentle female voice as an example, the virtual image corresponding to the candidate voice timbre can be a cartoon image of a girl, and the name can be Xiaomei. By displaying at least one candidate voice timbre in the resource editing interface, it is convenient for users to quickly select the voice timbre of the object in the initial media resource, which not only meets the user's personalized needs for voice timbre, but also improves the efficiency of human-computer interaction.

[0165] It should be noted that the present disclosure does not limit the execution order of step 303 and step 305 or step 304 and step 305. In this embodiment, only the sequential execution of step 303 and step 305 or the sequential execution of step 304 and step 305 is used as an example for explanation; in other embodiments, the sequence of step 305 and step 303 or step 305 and step 304 may also be followed.

[0166] In some embodiments, since the voice timbre of an object is related to the object type and the voice text of the object, after acquiring the voice text, the terminal can recommend at least one candidate voice timbre suitable for the object to the user based on the initial media resources and the voice text of the object. Accordingly, the terminal can screen a plurality of candidate voice timbres with different pronunciation characteristics based on the initial media resources and the voice text, and obtain at least one candidate voice timbre suitable for the object in the initial media resources; then, the terminal can display at least one candidate voice timbre in the resource editing interface for the user to select. Accordingly, the process of the terminal acquiring the voice timbre of the object includes: in response to the selection operation of any candidate voice timbre, acquiring the selected candidate voice timbre, that is, the terminal can determine the candidate voice timbre as the voice timbre of the object in the initial media resource. By recommending at least one candidate voice timbre suitable for the object to the user based on the initial media resources and the voice text of the object in the initial media resources, the user does not need to manually search for the voice timbre suitable for the object, thereby improving the efficiency of acquiring the voice timbre and improving the efficiency of human-computer interaction.

[0167] In some embodiments, when the initial media resource includes at least one object, the voice text can include the voice text of the at least one object. Accordingly, the terminal screens a plurality of candidate voice timbres based on the initial media resource and the voice text, and the process of obtaining at least one candidate voice timbre includes: the terminal performs image recognition on the initial media resource, and can obtain the object information of at least one object in the initial media resource. Then, the terminal can determine the dialogue role corresponding to each object based on the object information of at least one object and the voice text of at least one object. Finally, for any object, the terminal can screen a plurality of candidate voice timbres based on the dialogue role corresponding to the object, and obtain at least one candidate voice timbre matched by the object. Among them, when the number of at least one object is multiple, the voice text is the voice text generated by the dialogue of multiple objects. In the dialogue process, different objects play different dialogue roles, such as the dialogue role played by object A in the dialogue process is a teacher, and the dialogue role played by object B in the dialogue process is a student. Since different dialogue roles are suitable for different voice timbres, in order to determine the candidate voice timbre suitable for the object in the initial media resource, the terminal can first determine the dialogue role played by the object in the dialogue process based on the object information and the voice text of the object, and then can screen a variety of candidate voice timbres according to the dialogue role corresponding to the object, and obtain at least one candidate voice timbre matching the object. For example, in the case where the dialogue role corresponding to the object is a teacher, the terminal can screen out candidate voice timbre with relatively mature pronunciation characteristics as the candidate voice timbre matching the object. In the case where the dialogue role corresponding to the object is a student, the terminal can screen out candidate voice timbre with relatively young pronunciation characteristics as the candidate voice timbre matching the object. By recommending a candidate voice timbre suitable for the object to the user according to the dialogue role played by the object in the initial media resource during the dialogue process, it is possible to improve the efficiency of human-computer interaction while improving the accuracy of at least one candidate voice timbre recommended by the terminal to the user, so that the user can quickly select a voice timbre suitable for the object in the initial media resource.

[0168] In some embodiments, the terminal can determine the dialogue role played by the object based on the object relationship between the objects in the initial media resource. Accordingly, the terminal can determine the object relationship of at least one object based on the object information of at least one object and the voice text of at least one object; then, for any object, the terminal can determine the dialogue role corresponding to the object based on the object information and object relationship of the object. Among them, the object relationship can include friends, relatives, lovers, classmates, teachers and students, employers and employees, etc. Since the object information corresponding to different object relationships and the dialogue content of the objects are different, the terminal can determine the object relationship of the objects in the initial media resource based on the object information and the voice text of the objects in the initial media resource. For example, the terminal can determine that object A is a young male based on the object information and voice text, and object B is a young female and the voice text is "Me too", then the terminal can determine that the object relationship between object A and object B is a couple. After determining the object relationship, the terminal can determine the dialogue role played by the object in the dialogue process according to the object information and object relationship of any object. For example, in the case where the object relationship is a couple and the object information of object A is a young male, the terminal can determine that the object role played by object A in the object process is a boyfriend. By determining the dialogue role played by the object based on the object relationship between the objects in the initial media resources, the accuracy of the dialogue role determined by the terminal can be improved, and the accuracy of the candidate voice timbre suitable for the dialogue role determined by the terminal can be improved, thereby improving the efficiency of human-computer interaction while allowing users to quickly select a voice timbre suitable for the object in the initial media resources.

[0169] In some embodiments, after acquiring the voice text, for any object, the terminal can also identify the voice text of the object and determine the text theme of the voice text. Accordingly, the process of the terminal screening multiple candidate voice timbres based on the dialogue role corresponding to the object can be replaced by: the terminal screens multiple candidate voice timbres based on the dialogue role corresponding to the object and the text theme of the voice text. Among them, the text theme refers to the form and structure of the text, including prose, poetry, novels, etc. For example, in the case where the voice text is a text with a certain rhythm and rhythm, the terminal can determine that the text theme of the voice text is poetry. Further, in the case where the structure of the voice text is five words per sentence, a total of four sentences, the terminal can determine that the text theme of the voice text is a five-character quatrain in poetry. Since the voice timbres adapted to texts of different text themes are different, in order to determine the candidate voice timbre suitable for the voice text of the object in the initial media resource, the terminal determines the text theme of the voice text based on the voice text of the object, and then can screen multiple candidate voice timbres based on the text theme and the dialogue role played by the object in the dialogue process, and obtain at least one candidate voice timbre that matches the object and the text theme of the voice text of the object. For example, when the text subject of the object's voice text is poetry and the object's dialogue role is a teacher, the terminal can screen out candidate voice timbres with relatively mature pronunciation characteristics and suitable for reading poetry as candidate voice timbres for the object to match. By recommending candidate voice timbres suitable for the object to the user based on the dialogue role played by the object in the dialogue process in the initial media resource and the text subject of the object's voice text, the efficiency of human-computer interaction can be improved while further improving the accuracy of at least one candidate voice timbre recommended by the terminal to the user, so that the user can quickly select a voice timbre suitable for the object in the initial media resource.

[0170] In some embodiments, the first object is the user object currently logged in by the terminal, and when at least one candidate voice timbre displayed on the resource editing interface includes the voice timbre of the first object, the process of the terminal obtaining the voice timbre of the first object includes: in response to the timbre adding operation on the first object in the resource editing interface, the terminal displays the timbre recording interface; then, in response to the voice input operation in the timbre recording interface, the terminal obtains the voice signal input by the first object; finally, the terminal can obtain the voice timbre of the first object based on the voice signal. Among them, the timbre adding operation on the first object refers to the operation of adding the voice timbre of the first object. Figure 5 As shown in (a) of FIG. 1 , the resource editing interface also displays a timbre adding control 505. In response to a triggering operation on the timbre adding control 505, the terminal can display an object selection interface. For example, Figure 6 FIG. 1 is a schematic diagram showing a process of recording a tone color according to an exemplary embodiment. Figure 6As shown in (a) of FIG. 6 , the object selection interface 601 displays two timbre recording objects to be selected, including a first object and a second object. In response to the selection operation of the first object, the terminal can display the following Figure 6 The timbre recording interface 602 shown in (b) in FIG. The timbre recording interface 602 is used to record the voice signal of the first object. The timbre recording interface 602 also displays a timbre recording control 603. The voice input operation in the timbre recording interface 602 is a triggering operation on the timbre recording control 603. When the user successfully reads out the preset text content "xxxxxx" displayed in the timbre recording interface 602, the terminal can obtain the voice signal of the first object, and then can obtain the voice timbre of the first object based on the voice signal of the first object. By recording the user's own voice timbre, not only can the user select his own voice timbre as the voice timbre of the object in the initial media resource when the object in the initial media resource is the user himself, thereby improving the adaptability of the object and the voice timbre, but also can enable the user to obtain a voice with the user's voice characteristics without manually dubbing the object, thereby reducing the operating cost of dubbing the object in the initial media resource and improving the efficiency of human-computer interaction.

[0171] In some embodiments, after acquiring the voice timbre of the first object, the terminal can Figure 6 The page shown in (c) in FIG. 1 displays a prompt message, prompting the user that the voice tone has been successfully recorded. The terminal can also Figure 6 The resource editing interface 604 shown in (d) displays the voice timbre “own timbre” of the first object, so that the user can select his own voice timbre from the candidate voice timbre as the voice timbre of the object in the initial media resource.

[0172] In some embodiments, the second object is an object that has an association relationship with the first object, and the first object is the user object currently logged in at the terminal. When at least one candidate voice timbre displayed on the resource editing interface includes the voice timbre of the second object, the process of the terminal acquiring the voice timbre of the second object includes: in response to the timbre adding operation of the second object in the resource editing interface, the terminal can send a timbre entry link to the second object; then, when the second object successfully enters the voice content, the terminal can obtain the voice signal input by the second object in the timbre entry interface corresponding to the timbre entry link; finally, the terminal can acquire the voice timbre of the second object based on the voice signal. Among them, the association relationship can be a relationship in which the first object and the second object are concerned with each other. Or, the first object and the second object are in a close relationship with each other. The close relationship includes relationships that can be set in applications that support instant messaging, such as friend relationships, family relationships, lover relationships, and classmate relationships. The timbre adding operation to the second object refers to the operation of adding the voice timbre of the second object. Such as Figure 5 As shown in (a) of FIG. 1 , the resource editing interface also displays a timbre adding control 505. In response to a triggering operation on the timbre adding control 505, the terminal can display an object selection interface. For example, Figure 7 FIG. 1 is a schematic diagram of another timbre recording process according to an exemplary embodiment. Figure 7 As shown in (a) of FIG. 7 , the object selection interface 701 displays two timbre recording objects to be selected, including a first object and a second object. In response to the selection operation of the second object, the terminal can display the following Figure 7 The object invitation interface 702 shown in (b) of FIG. 7 shows at least one second object associated with the first object. In response to the selection operation of any second object, that is, in response to the triggering operation of the invitation control 703 of "Friend A" in the object invitation interface 702, the terminal can send a timbre recording link to the selected second object. Figure 7 As shown in (c) in the figure, the terminal can display the sent timbre recording link 704 in the conversation interface between the first object and the second object. When the second object successfully records the voice content, the terminal can obtain the voice signal input by the second object in the timbre recording interface corresponding to the timbre recording link 704, and then can obtain the voice timbre of the second object based on the voice signal of the second object. By recording the voice timbre of other users who have an associated relationship with the user, not only can the user select the voice timbre of the friend as the voice timbre of the object in the initial media resource when the object in the initial media resource is the user's friend, thereby improving the adaptability of the object and the voice timbre, but also can enable the user to obtain a voice with the friend's voice characteristics without inviting the friend to manually dub the object, thereby reducing the operating cost of dubbing the object in the initial media resource and improving the efficiency of human-computer interaction.

[0173] In some embodiments, after acquiring the voice timbre of the second object, the terminal can Figure 7 The prompt information is displayed in the conversation interface shown in (c) of FIG. 1 to prompt the user that the voice timbre of the second object has been successfully recorded. The terminal can also Figure 7 The resource editing interface 705 shown in (d) displays the voice timbre “object A timbre” of the second object, so that the user can select the voice timbre of the second object from the candidate voice timbre as the voice timbre of the object in the initial media resource.

[0174] In some embodiments, the terminal can perform timbre analysis on the collected voice signal to obtain the voice timbre corresponding to the voice signal. Alternatively, the terminal can send the collected voice signal to the server, and the server performs timbre analysis on the voice signal to obtain the voice timbre corresponding to the voice signal. The terminal can receive the voice timbre returned by the server.

[0175] In step 306, the terminal generates a target media resource based on the voice text, the voice timbre and the initial media resource, the target media resource includes a target voice and the initial media resource, the content of the target voice is the voice text, and the timbre of the target voice is the voice timbre.

[0176] In the embodiment of the present disclosure, the process of the terminal generating the target media resource based on the voice text, the voice timbre and the initial media resource is the same as the above step 203, which will not be repeated here.

[0177] In step 307, the terminal displays the target media resource.

[0178] In the embodiments of the present disclosure, Figure 5 As shown in (a) of FIG. 1 , after obtaining the target media resource, in response to the triggering operation of adding the resource control 503, the terminal can Figure 5 The target media resource is displayed in the conversation interface 504 shown in (b) so that the user can interact with other users using the target media resource.

[0179] In step 308, in response to a sending operation on the target media resource in the conversation interface, the terminal displays the target media resource and a voice play control of the target media resource in the conversation interface.

[0180] In the disclosed embodiment, Figure 8 is a schematic diagram of a conversation interface according to an exemplary embodiment. Figure 8 As shown, the conversation interface 801 includes an information input area and an information display area. The information input area displays the target media resource to be sent. In response to the sending operation of the target media resource in the information input area, the terminal can display the target media resource and the voice playback control 802 of the target media resource in the information display area.

[0181] In step 309, in response to the triggering operation on the voice playing control, the terminal plays the target voice in the target media resource.

[0182] In the embodiments of the present disclosure, Figure 8 As shown, in response to the triggering operation of the voice playback control 802, the terminal can play the target voice in the target media resource in the conversation interface, so that the user can feel the information expressed by the target media resource from a visual dimension, and can also feel the information expressed by the target media resource from an auditory dimension, thereby enriching the interactive effect of the user using the target media resource to communicate and enhancing the interactivity between users.

[0183] The disclosed embodiment provides a resource display method, which obtains the voice text and voice timbre configured by the user for the initial media resource in the resource editing interface, so that the terminal can generate a target media resource that combines the voice resource and the media resource based on the voice text, the voice timbre and the initial media resource, that is, the target media resource can express the user's emotions from both the visual dimension and the auditory dimension, thereby enriching the emotional expression dimension of the target media resource, and further enriching the interactive effect of the user using the target media resource to communicate, thereby enhancing the interactivity between users.

[0184] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0185] Fig. 9 FIG. 1 is a block diagram of a resource display device according to an exemplary embodiment. Figure 8 As shown, the device includes: a first display unit 901 , a first acquisition unit 902 , and a generation unit 903 .

[0186] The first display unit 901 is configured to display a resource editing interface in response to a trigger operation on a resource creation portal, wherein the resource editing interface displays an initial media resource;

[0187] A first acquisition unit 902 is configured to acquire a voice text and a voice timbre configured for the initial media resource in response to a voice configuration operation on the initial media resource;

[0188] The generating unit 903 is configured to generate a target media resource based on the voice text, the voice timbre and the initial media resource, wherein the target media resource includes a target voice and the initial media resource, the content of the target voice is the voice text, and the timbre of the target voice is the voice timbre;

[0189] The first display unit 901 is further configured to display the target media resource.

[0190] In some embodiments, the first display unit 901 is configured to display multiple candidate media resources in response to a trigger operation on a resource creation portal; in response to a selection operation on any candidate media resource, display an initial media resource in a resource editing interface based on the selected candidate media resource.

[0191] In some embodiments, the first display unit 901 is configured to display the selected candidate media resource on the resource editing interface when the selected candidate media resource is an image resource; or, when the selected candidate media resource is a video resource, edit the selected candidate media resource to obtain a media resource containing at least one video frame; and display the media resource containing at least one video frame on the resource editing interface.

[0192] In some embodiments, the first acquisition unit 902 is configured to acquire input voice text in response to an input operation in the resource editing interface; or to perform image recognition on the initial media resource to obtain the voice text.

[0193] In some embodiments, the first acquisition unit 902 is configured to perform image recognition on the initial media resource to obtain an image recognition result of the initial media resource, the image recognition result includes at least one of text information, object information and emotion information, the text information is used to indicate the image text in the initial media resource, the object information is used to indicate the object information of the object in the initial media resource, and the emotion information is used to indicate the emotion represented by the object in the initial media resource; based on the image recognition result, speech text is generated.

[0194] In some embodiments, the first acquisition unit 902 is configured to generate voice text based on text information when the initial media resource contains image text; or to generate voice text based on at least one of object information and emotion information when the initial media resource does not contain image text.

[0195] In some embodiments, the resource editing interface displays at least one candidate voice timbre and image information corresponding to each candidate voice timbre, and the image information includes a virtual image corresponding to the candidate voice timbre and a name of the virtual image;

[0196] The first acquisition unit 902 is configured to acquire the selected candidate voice timbre in response to a selection operation on any candidate voice timbre.

[0197] In some embodiments, Fig.10 is a block diagram of another resource display device according to an exemplary embodiment. Fig.10 , the device further comprises:

[0198] A screening unit 904 screens a plurality of candidate voice timbres based on the initial media resources and the voice text to obtain at least one candidate voice timbre;

[0199] A first display unit 901 is configured to display at least one candidate voice timbre on a resource editing interface;

[0200] The first acquisition unit 902 is configured to acquire the selected candidate voice timbre in response to a selection operation on any candidate voice timbre.

[0201] In some embodiments, where the initial media resource includes at least one object, the spoken text includes the spoken text of the at least one object;

[0202] The screening unit 904 is configured to perform image recognition on the initial media resource to obtain object information of at least one object in the initial media resource; determine the dialogue role corresponding to each object based on the object information of at least one object and the voice text of at least one object; for any object, based on the dialogue role corresponding to the object, screen multiple candidate voice timbres to obtain at least one candidate voice timbre matching the object.

[0203] In some embodiments, the screening unit 904 is configured to determine the object relationship of at least one object based on the object information of at least one object and the voice text of at least one object; for any object, determine the dialogue role corresponding to the object based on the object information and the object relationship of the object.

[0204] In some embodiments, see Fig.10 , the device further comprises:

[0205] The determination unit 905 is configured to recognize the speech text of any object and determine the text subject of the speech text;

[0206] The screening unit 904 is configured to screen a plurality of candidate voice timbres based on the dialogue role corresponding to the object and the text subject matter of the voice text.

[0207] In some embodiments, at least one candidate voice timbre includes a voice timbre of a first object, the first object being a user object currently logged in by the terminal;

[0208] Continue to see Fig.10 , the device further comprises:

[0209] The second display unit 906 is configured to display a timbre input interface in response to a timbre adding operation on the first object in the resource editing interface;

[0210] The second acquisition unit 907 is configured to acquire a voice signal input by the first object in response to a voice input operation in the timbre recording interface; and acquire the voice timbre of the first object based on the voice signal.

[0211] In some embodiments, at least one candidate voice timbre includes a voice timbre of a second object, the second object is an object associated with the first object, and the first object is a user object currently logged in by the terminal;

[0212] Continue to see Fig.10 , the device further comprises:

[0213] The sending unit 908 is configured to send a timbre entry link to the second object in response to a timbre adding operation on the second object in the resource editing interface;

[0214] The second acquisition unit 907 is further configured to acquire the voice signal input by the second object in the timbre recording interface corresponding to the timbre recording link when the second object successfully records the voice content; and acquire the voice timbre of the second object based on the voice signal.

[0215] In some embodiments, see Fig.10 , the device further comprises:

[0216] The third display unit 909 is configured to display the target media resource and the voice playback control of the target media resource in the conversation interface in response to the sending operation of the target media resource in the conversation interface;

[0217] The playing unit 910 is configured to play the target voice in the target media resource in response to a triggering operation on the voice playing control.

[0218] The disclosed embodiment provides a resource display device, which obtains the voice text and voice timbre configured by the user for the initial media resource in the resource editing interface, so that the terminal can generate a target media resource that combines the voice resource and the media resource based on the voice text, the voice timbre and the initial media resource, that is, the target media resource can express the user's emotions from both the visual dimension and the auditory dimension, thereby enriching the emotional expression dimension of the target media resource, and further enriching the interactive effect of the user using the target media resource to communicate, thereby enhancing the interactivity between users.

[0219] It should be noted that the resource display device provided in the above embodiment is only illustrated by the division of the above functional units. In actual applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the terminal is divided into different functional units to complete all or part of the functions described above. In addition, the resource display device provided in the above embodiment and the resource display method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0220] Regarding the resource display device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0221] Fig.11 It is a block diagram of a terminal according to an exemplary embodiment. Generally, the terminal 1100 includes: a processor 1101 and a memory 1102 .

[0222] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0223] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one program code, which is used to be executed by the processor 1101 to implement the resource display method provided in the method embodiment of the present disclosure.

[0224] In some embodiments, the terminal 1100 may further optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, the memory 1102 and the peripheral device interface 1103 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1103 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107 and a power supply 1108.

[0225] The peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0226] The radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1104 may also include circuits related to NFC (Near Field Communication), which is not limited in the present disclosure.

[0227] The display screen 1105 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1105 is a touch display screen, the display screen 1105 also has the ability to collect touch signals on the surface or above the surface of the display screen 1105. The touch signal can be input to the processor 1101 as a control signal for processing. At this time, the display screen 1105 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1105 can be one, and the front panel of the terminal 1100 is set; in other embodiments, the display screen 1105 can be at least two, which are respectively set on different surfaces of the terminal 1100 or are folded; in some other embodiments, the display screen 1105 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 1100. Even, the display screen 1105 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0228] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0229] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1101 for processing, or input them into the radio frequency circuit 1104 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1100. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0230] The power supply 1108 is used to power various components in the terminal 1100. The power supply 1108 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1108 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0231] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the terminal 1100, and the terminal 1100 may include more or fewer components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0232] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1102 including instructions, and the instructions can be executed by a processor 1101 of a terminal 1100 to complete the resource display method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0233] A computer program product includes a computer program, and the computer program implements the resource display method when executed by a processor.

[0234] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0235] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A resource display method, characterized in that: The method comprises: In response to a trigger operation on a resource creation entry, displaying a resource editing interface, wherein the resource editing interface displays an initial media resource; In response to the voice configuration operation on the initial media resource, acquiring the voice text and voice timbre configured for the initial media resource; Based on the speech text, the speech timbre and the initial media resource, a target media resource is generated, wherein the target media resource includes a target speech and the initial media resource, the content of the target speech is the speech text, and the timbre of the target speech is the speech timbre; The target media resource is displayed.

2. The resource display method according to claim 1, characterized in that: The displaying of a resource editing interface in response to a triggering operation on a resource creation entry includes: In response to a triggering operation on a resource creation entry, displaying a plurality of candidate media resources; In response to a selection operation on any candidate media resource, the initial media resource is displayed on the resource editing interface based on the selected candidate media resource.

3. The resource display method according to claim 2, characterized in that: The displaying the initial media resource on the resource editing interface based on the selected candidate media resource includes: In the case where the selected candidate media resource is an image resource, displaying the selected candidate media resource on the resource editing interface; or, In a case where the selected candidate media resource is a video resource, editing the selected candidate media resource to obtain a media resource including at least one video frame; The media resource including at least one video frame is displayed on the resource editing interface.

4. The resource display method according to claim 1, characterized in that: The process of acquiring the voice text includes: In response to an input operation in the resource editing interface, obtaining input voice text; or, Perform image recognition on the initial media resource to obtain the voice text.

5. The resource display method according to claim 4, characterized in that: The performing image recognition on the initial media resource to obtain the voice text includes: Performing image recognition on the initial media resource to obtain an image recognition result of the initial media resource, wherein the image recognition result includes at least one of text information, object information, and emotion information, wherein the text information is used to indicate image text in the initial media resource, the object information is used to indicate object information of an object in the initial media resource, and the emotion information is used to indicate an emotion represented by the object in the initial media resource; Based on the image recognition result, the speech text is generated.

6. The resource display method according to claim 5, characterized in that: The generating the speech text based on the image recognition result includes: In the case where the initial media resource includes image text, generating the voice text based on the text information; or, In a case where the initial media resource does not include image text, the voice text is generated based on at least one of the object information and the emotion information.

7. The resource display method according to claim 1, characterized in that: The resource editing interface displays at least one candidate voice timbre and image information corresponding to each candidate voice timbre, wherein the image information includes a virtual image corresponding to the candidate voice timbre and a name of the virtual image; The process of acquiring the voice timbre includes: In response to a selection operation on any candidate voice timbre, the selected candidate voice timbre is acquired.

8. The resource display method according to claim 1, characterized in that: After acquiring the voice text, the method further includes: Based on the initial media resource and the voice text, a plurality of candidate voice timbres are screened to obtain at least one candidate voice timbre; Displaying the at least one candidate voice timbre on the resource editing interface; The process of acquiring the voice timbre includes: In response to a selection operation on any candidate voice timbre, the selected candidate voice timbre is acquired.

9. The resource display method according to claim 8, characterized in that: In the case where the initial media resource includes at least one object, the speech text includes the speech text of the at least one object; The method of screening a plurality of candidate voice timbres based on the initial media resource and the voice text to obtain at least one candidate voice timbre comprises: Performing image recognition on the initial media resource to obtain object information of at least one object in the initial media resource; Determining a dialogue role corresponding to each object based on the object information of the at least one object and the voice text of the at least one object; For any object, the plurality of candidate voice timbres are screened based on the dialogue role corresponding to the object to obtain at least one candidate voice timbre matching the object.

10. The resource display method according to claim 9, characterized in that: The determining, based on the object information of the at least one object and the voice text of the at least one object, a dialogue role corresponding to each object includes: determining an object relationship of the at least one object based on the object information of the at least one object and the speech text of the at least one object; For any object, a dialogue role corresponding to the object is determined based on the object information and the object relationship of the object.

11. The resource display method according to claim 9, characterized in that: The method further comprises: For any object, recognize the speech text of the object and determine the text subject of the speech text; The screening of the plurality of candidate voice timbres based on the dialogue role corresponding to the object includes: The plurality of candidate voice timbres are screened based on the dialogue role corresponding to the object and the text subject matter of the voice text.

12. The resource display method according to claim 7 or 8, characterized in that: The at least one candidate voice timbre includes the voice timbre of a first object, where the first object is a user object currently logged in by the terminal; The method further comprises: In response to a timbre adding operation on the first object in the resource editing interface, displaying a timbre recording interface; In response to a voice input operation in the timbre recording interface, acquiring a voice signal input by the first object; Based on the voice signal, the voice timbre of the first object is acquired.

13. The resource display method according to claim 7 or 8, characterized in that: The at least one candidate voice timbre includes a voice timbre of a second object, the second object is an object associated with the first object, and the first object is a user object currently logged in by the terminal; The method further comprises: In response to a timbre adding operation on the second object in the resource editing interface, sending a timbre recording link to the second object; When the second object successfully records the voice content, obtaining the voice signal input by the second object in the timbre recording interface corresponding to the timbre recording link; Based on the voice signal, the voice timbre of the second object is acquired.

14. The resource display method according to claim 1, characterized in that: The method further comprises: In response to a sending operation on the target media resource in the conversation interface, displaying the target media resource and a voice playback control of the target media resource in the conversation interface; In response to a triggering operation on the voice playing control, the target voice in the target media resource is played.

15. A resource display device, characterized in that: The device comprises: A first display unit is configured to display a resource editing interface in response to a trigger operation on a resource creation portal, wherein the resource editing interface displays an initial media resource; A first acquisition unit is configured to acquire a voice text and a voice timbre configured for the initial media resource in response to a voice configuration operation on the initial media resource; A generating unit is configured to generate a target media resource based on the voice text, the voice timbre and the initial media resource, wherein the target media resource includes a target voice and the initial media resource, the content of the target voice is the voice text, and the timbre of the target voice is the voice timbre; The first display unit is further configured to display the target media resource.

16. A terminal, characterized in that: The terminal comprises: one or more processors; a memory for storing program code executable by the processor; The processor is configured to execute the program code to implement the resource display method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of a terminal, the terminal is enabled to execute the resource display method according to any one of claims 1 to 14.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the resource display method according to any one of claims 1 to 14 is implemented.

Citation Information

Patent Citations

  • Voice expression display method and device and voice expression generation method and device

    CN112910752A

  • Multimedia teaching resource determination method and device based on image processing and identification

    CN114519655A

  • Text processing method and device, electronic equipment, storage medium and program product

    CN117828010A

  • Expression interaction method and device, equipment and storage medium

    CN117834576A

  • Display control method, computer-readable recording medium, and computer device for displaying conversation response candidate for user speech input

    WO2019142976A1

Cited By

  • Multimedia resource marking method, multimedia resource display method and multimedia resource marking device

    CN120751205A