Instant interaction method and device, network equipment, medium and computer program product

Abstract

The embodiment of the invention provides an instant interaction method and device, network equipment, a medium and a computer program product, and relates to the technical field of meta universe. The method comprises the steps of presenting a second playing interface on a first playing interface, used for playing the meta-universe conference, of a first user in the proceeding process of the meta-universe conference; in the process of information interaction between the first user and the second user through the second playing interface, after first interaction information on the second playing interface is obtained, according to the emotional characteristics corresponding to the first interaction information, the first interaction information on the second playing interface is obtained; converting second interaction information responding to the first interaction information into response information conforming to the emotional characteristics; and displaying the response information on the second playing interface. By adopting the method, the privacy, the efficient instantaneity and the emotional effect of the session between the first user and the second user can be ensured, so that the information interaction process is more in line with the interaction scene of the numerous wisdom in the meta universe conference.

Description

Technical Field

[0001] The present application relates to the technical field of the metaverse, and in particular, to an instant interaction method, device, network device, medium, and computer program product. Background Art

[0002] In the current metaverse conference scenario, digital humans have been widely used. For example, digital human assistants can be used for intelligent question answering; for the digital twin corresponding to oneself, its actions can be controlled, etc. However, the current metaverse conferences usually only have a multi-person interaction function and lack privacy. When it is necessary to have a conversation with other users for personal matters in the conference space, it is often necessary to unmute and shout, which affects the conference experience of other conference participants. And using an online private communication method has the problem of not being able to achieve emotional interaction and not matching the digital human interaction scenario in the metaverse conference. Summary of the Invention

[0003] The object of the technical solution of the present application is to provide an instant interaction method, device, network device, medium, and computer program product, which are used to realize private communication and emotional interaction among a small number of users in a metaverse conference, and ensure that the information interaction process is more in line with the digital human interaction scenario in the metaverse conference.

[0004] One embodiment of the present application provides an instant interaction method, which includes:

[0005] During the progress of a metaverse conference, a second playback interface is presented on the first playback interface used by a first user to play the metaverse conference;

[0006] During the process of the first user and the second user interacting information through the second playback interface, after obtaining the first interaction information on the second playback interface, according to the emotional characteristics corresponding to the first interaction information, the second interaction information in response to the first interaction information is converted into a response message that conforms to the emotional characteristics;

[0007] The response message is displayed on the second playback interface.

[0008] Optionally, in the instant interaction method, the method further includes:

[0009] After obtaining the information sent by the message sending user among the first user and the second user on the second playback interface, the first interaction information is obtained;

[0010] Performing emotional analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information;

[0011] After obtaining the response information of the message respondent among the first user and the second user in response to the first interaction information, obtain the second interaction information.

[0012] Optionally, the instant interaction method, wherein the method further includes:

[0013] Display the emotion information corresponding to the emotion feature in the form of bullet comments on the second playback interface.

[0014] Optionally, the instant interaction method, wherein performing sentiment analysis on the first interaction information to obtain the emotion feature corresponding to the first interaction information includes:

[0015] Perform word segmentation on the first interaction information to obtain multiple word segments and corresponding parts of speech;

[0016] Generate a word vector matrix according to the multiple word segments and corresponding parts of speech;

[0017] Obtain the global semantic feature vector and local semantic feature vector of the first interaction information according to the word vector matrix;

[0018] Perform sentiment analysis on the fused feature information after fusing the global semantic feature vector and the local semantic feature vector to obtain the emotion feature corresponding to the first interaction information.

[0019] Optionally, the instant interaction method, wherein obtaining the global semantic feature vector and local semantic feature vector of the first interaction information according to the word vector matrix includes:

[0020] Input the word vector matrix into the TextCNN model to obtain the local semantic feature vector;

[0021] Input the word vector matrix into the BiGRU model to obtain global correlation information;

[0022] Adopt a self-attention learning mechanism to perform relevance and / or logical analysis on the global correlation information to obtain the global semantic feature vector.

[0023] Optionally, the instant interaction method, wherein during the metaverse conference, presenting a second playback interface on the first playback interface used by the first user to play the metaverse conference includes:

[0024] During the metaverse conference, in response to a first request triggered by the first digital human corresponding to the first user on the first playback interface, establish the second playback interface on the first playback interface;

[0025] Wherein, the method further includes:

[0026] Determine the second user according to the second digital human selected by the first digital human on the first playback interface or the second playback interface;

[0027] Establish the second playback interface on the first playback interface used by the second user to play the metaverse meeting.

[0028] Optionally, in the instant interaction method, the method further includes:

[0029] When establishing the second playback interface, generate the number information and login password of the second playback interface;

[0030] Obtain a second request sent by a third digital human corresponding to a third user, where the second request is used to request to join the interaction between the first user and the second user, and the second request includes the number information and the login password;

[0031] When the verification of the number information and login password of the second playback interface is successful, establish the second playback interface on the first playback interface used by the third user to play the metaverse meeting, and display the interaction information among the first user, the second user, and the third user on the second playback interface.

[0032] One embodiment of the present application further provides an instant interaction device, including:

[0033] An interface establishment module, configured to present a second playback interface on a first playback interface used by a first user to play the metaverse meeting during the metaverse meeting;

[0034] An information conversion module, configured to, during the process of the first user and the second user interacting information through the second playback interface, after obtaining first interaction information on the second playback interface, convert second interaction information in response to the first interaction information into a response information that conforms to the emotional characteristics according to the emotional characteristics corresponding to the first interaction information;

[0035] A display module, configured to display the response information on the second playback interface.

[0036] One embodiment of the present application further provides a network device, including a processor, a memory, and a program stored on the memory and executable on the processor, where the program, when executed by the processor, implements the instant interaction method described in any one of the above.

[0037] One embodiment of the present application further provides a readable storage medium, where a program is stored on the readable storage medium, and the program, when executed by a processor, implements the steps in the instant interaction method described in any one of the above.

[0038] One embodiment of the present application further provides a computer program product, which includes computer instructions that, when executed by a processor, implement the steps in the instant interaction method described in any one of the above.

[0039] At least one of the above technical solutions of the present application has the following beneficial effects:

[0040] In the instant interaction method described in the embodiments of the present application, on the first playback interface presented during the metaverse meeting, the second playback interface is displayed in the form of a "small window interface" to implement the interaction between the first user and the second user, that is, to implement the private information interaction between a small number of digital humans in the metaverse meeting; in addition, by performing sentiment analysis on the first interaction information and, according to the obtained emotion characteristics, converting the style of the second interaction information responded by the message answering user into a response message that conforms to the emotion characteristics, making the interaction between digital avatars more emotional, so as to ensure the privacy, high-efficiency instantaneity and emotion of the conversation between the digital humans corresponding to the first user and the second user, and making the information interaction process more in line with the digital human interaction scenario in the metaverse meeting. Description of the Drawings

[0041] Figure 1 It is a schematic flowchart of one implementation manner of the instant interaction method described in the embodiments of the present application;

[0042] Figure 2 It is a schematic diagram of one display form of the first playback interface and the second playback interface in the embodiments of the present application;

[0043] Figure 3 It is a schematic diagram of one display form of the second playback interface in the embodiments of the present application;

[0044] Figure 4 It is a schematic structural diagram of one implementation manner of the instant interaction device described in the embodiments of the present application. Detailed Embodiments

[0045] To make the technical problems, technical solutions and advantages to be solved by the present application clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0046] To solve the problem in the prior art that in a metaverse conference, only the function of multi-person interaction is available and private communication between a small number of users cannot be achieved, the embodiments of the present application provide an instant interaction method. By presenting a second playback interface on the first playback interface played in the metaverse conference and displaying the interaction information between the first user and the second user on the second playback interface, private communication between the first user and the second user is realized. In addition, by performing sentiment analysis on the first interaction information and, according to the obtained emotion characteristics, converting the style of the second interaction information responded to by the message respondent into a response message that conforms to the emotion characteristics, the interaction between the digital avatars becomes more emotional, so as to ensure the privacy, high-efficiency instantaneity, and emotion of the conversation between the digital humans corresponding to the first user and the second user, and ensure that the information interaction process is more in line with the digital human interaction scenario in the metaverse conference.

[0047] As Figure 1 shown in the flowchart of one embodiment of the instant interaction method described in the embodiments of the present application. In this embodiment, the instant interaction method includes:

[0048] S110, during the metaverse conference, present a second playback interface on the first playback interface used by the first user to play the metaverse conference;

[0049] S120, during the process of the first user and the second user interacting information through the second playback interface, after obtaining the first interaction information on the second playback interface, convert the second interaction information responding to the first interaction information into a response message that conforms to the emotion characteristics according to the emotion characteristics corresponding to the first interaction information;

[0050] S130, display the response message on the second playback interface.

[0051] In the embodiments of the present application, the first playback interface is the display interface presented in the metaverse conference space, and the second playback interface is the "small window interface" presented in the metaverse conference space. By using the instant interaction method described in this embodiment, the second playback interface is displayed in the form of a "small window interface" on the first playback interface presented during the playback of the metaverse conference to realize the interaction between the first user and the second user. Among them, each user corresponds to a digital human during the metaverse conference, and the interaction between the first user and the second user is also to realize the private information interaction between a small number of digital humans in the metaverse conference, so as to ensure the privacy and high-efficiency instantaneity of the conversation between the digital humans corresponding to the first user and the second user.

[0052] In addition, by performing sentiment analysis on the first interaction information on the second playback interface and, based on the emotional characteristics obtained from the analysis, converting the style of the second interaction information responded to by the message-replying user into a response message that conforms to the emotional characteristics, the interaction between the digital avatars becomes more emotional, so as to ensure the privacy, high-efficiency instantaneity, and emotionality of the conversation between the digital humans corresponding to the first user and the second user, and ensure that the information interaction process is more in line with the digital human interaction scenario in the metaverse conference.

[0053] In an embodiment of the present application, optionally, in step S110, presenting a second playback interface on the first playback interface used by the first user to play the metaverse conference includes:

[0054] During the metaverse conference, in response to a first request triggered by the first digital human corresponding to the first user on the first playback interface, establishing the second playback interface on the first playback interface;

[0055] Wherein, the method further includes:

[0056] Determining the second user according to the second digital human selected by the first digital human on the second playback interface;

[0057] Establishing the second playback interface on the first playback interface used by the second user to play the metaverse conference.

[0058] Adopting the method of this embodiment, the first digital human can be the initiating end for triggering the establishment of the second playback interface, initiate a first request on the first playback interface presented in the metaverse conference, in response to this first request, present the second playback interface on the first playback interface, and the first digital human can select a second digital human on the second playback interface for private information interaction. The process of triggering the establishment of the second playback interface for private interaction is simple and convenient, and enables the private interaction between the first digital human and the second digital human to have high-efficiency instantaneity.

[0059] Specifically, in the metaverse conference scenario, multiple participating users in the real world, such as the first user A and the second user B, enter the virtual metaverse conference space in the form of digital avatars. The interface presented by this metaverse conference space is the first playback interface. Optionally, the name of this first playback interface can be "Main Conference"; the digital avatar corresponding to the first user A is the first digital human (or called digital human A), and the digital avatar corresponding to the second user B is the second digital human (or called digital human B).

[0060] In the metaverse conference space, that is, during the metaverse conference, when the first digital human needs to communicate with the second digital human in real time, a first request is initiated on the first playback interface. In one implementation, optionally, a first icon is displayed on the first playback interface, and this first icon is used to trigger real-time communication. This first icon can also be referred to as the "meeting window" icon. With this implementation, when a click operation on this first icon is detected, it is determined that the first request triggered by the first digital human on the first playback interface is obtained.

[0061] In one implementation, optionally, the method further includes:

[0062] After obtaining the first request, a digital human selection list is presented on the first playback interface or the second playback interface.

[0063] Among them, with this implementation, according to the selection operation of the first digital human on this digital human selection list, the second digital human selected by the first digital human can be determined, thereby determining the corresponding second user.

[0064] In one implementation, optionally, when a digital human selection list is presented on the first playback interface after obtaining the first request, after detecting the selection operation of the first digital human on this digital human selection list, according to the second digital human selected by this selection operation, a second playback interface is generated, that is, a "window interface" is generated in the metaverse conference interface, and the first digital human and the second digital human are simultaneously pulled into this "window interface", as Figure 2 shown, so that this second playback interface is respectively presented on the first playback interface for playing the metaverse conference corresponding to the first user corresponding to the first digital human and the second user corresponding to the second digital human.

[0065] In an embodiment of the present application, in one implementation, optionally, the method further includes:

[0066] When establishing the second playback interface, number information and a login password for the second playback interface are generated;

[0067] A second request sent by a third digital human corresponding to a third user is obtained. The second request is used to request to join the interaction between the first user and the second user, and the number information and the login password are included in the second request;

[0068] When the verification of the number information and the login password of the second playback interface is successful, the second playback interface is established on the first playback interface for the third user to play the metaverse conference, and the interaction information among the first user, the second user, and the third user is displayed on the second playback interface.

[0069] In an embodiment of the present application, optionally, when establishing a second playback interface for presenting interaction information between a first user and a second user, number information and a login password for the second playback interface are generated simultaneously and saved in a database.

[0070] In one implementation, optionally, when a first digital human, which is the initiating end for triggering the establishment of the second playback interface, does not select a third user to join the interaction information, the third digital human corresponding to the third user can send a second request carrying the number information and the login password of the second playback interface to request to join the interaction between the first user and the second user; according to the second request, a server interface can be called to verify the number information and the login password of the second playback interface saved in the database with the number information and the login password in the second request. If the verification passes, a second playback interface can be established on the first playback interface used by the third user to play the metaverse meeting, and the interaction information between the first user, the second user, and the third user can be displayed on the second playback interfaces corresponding to the first user, the second user, and the third user, enabling the third digital human to join the interaction process of the second playback interface.

[0071] In one implementation of the present application, optionally, the method further includes:

[0072] Synchronously display the meeting information played on the first playback interface on the second playback interface.

[0073] In an embodiment of the present application, on the second playback interface, each digital human participating in the information interaction can perform interaction activities that are in the same process as the "main meeting" of the metaverse meeting but do not interfere with the process of the "main meeting". Optionally, the meeting information played on the first playback interface presenting the "main meeting" can be synchronously displayed on the second playback interface to achieve the same meeting functions as the "main meeting".

[0074] In one implementation, optionally, the first digital human, which is the initiating end for triggering the establishment of the second playback interface, can also customize the content displayed on the second playback interface to achieve on-demand setting of the content played on the second playback interface and enhance the flexibility of the interaction information between digital humans.

[0075] By adopting the instant interaction method described in the above implementation, by presenting the second playback interface on the first playback interfaces used by the first user corresponding to the first digital human and the second user corresponding to the second digital human to play the metaverse meeting respectively, that is, presenting a "small window interface" in the virtual metaverse meeting space, instant interaction between digital avatars is realized to enhance the immersive experience in the metaverse meeting scenario.

[0076] In an embodiment of the present application, in one implementation manner, optionally, the method further includes:

[0077] After obtaining the information sent by the message sending user among the first user and the second user on the second playback interface, obtain the first interaction information;

[0078] Perform sentiment analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information;

[0079] After obtaining the response information of the message replying user among the first user and the second user in response to the first interaction information, obtain the second interaction information.

[0080] Using the instant interaction method of this implementation manner, on the basis of realizing instant interaction between digital avatars by presenting a "small window interface" through a virtual metaverse conference space, perform sentiment analysis on the first interaction information sent by the message sending end to obtain the emotional characteristics corresponding to the first interaction information. According to the analyzed emotional characteristics, convert the second interaction information responded by the message replying user into a response message that conforms to the emotional characteristics, and display the response message on the second playback interface. Using this processing method makes the interaction between digital avatars more emotional and adds a touch of humanity to the interaction between digital humans in the virtual world.

[0081] In an embodiment of the present application, the message sending user is any one of the first user and the second user, that is, any user who sends an instant message, and the message replying user is the other user among the first user and the second user except the message sending user, that is, the user who responds to the message sent by the message sending user and sends a response message.

[0082] In one implementation manner, optionally, performing sentiment analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information includes:

[0083] Perform word segmentation processing on the first interaction information to obtain multiple word segments and corresponding part-of-speech tags;

[0084] Generate a word vector matrix according to the multiple word segments and corresponding part-of-speech tags;

[0085] Obtain the global semantic feature vector and local semantic feature vector of the first interaction information according to the word vector matrix;

[0086] Perform sentiment analysis on the fused feature information after fusing the global semantic feature vector and the local semantic feature vector to obtain the emotional characteristics corresponding to the first interaction information.

[0087] In one implementation, optionally, perform word segmentation on the first interaction information to obtain multiple segmented words and corresponding parts of speech, including:

[0088] Use the Jieba word segmentation tool to perform word segmentation and part-of-speech tagging on the first interaction information to obtain multiple segmented words and corresponding parts of speech;

[0089] For example, when setting the length of the first interaction information to n, the multiple segmented words W obtained by splitting are: W = {w(1), w(2),..., w(n)}, and the corresponding parts of speech M are M = {m(1), m(2),..., m(n)}.

[0090] In one implementation, optionally, generate a word vector matrix according to the multiple segmented words and corresponding parts of speech, including:

[0091] Concatenate the multiple segmented words and corresponding parts of speech to construct a segmented word sequence pair;

[0092] Use the Glove model to perform word vector training on the constructed segmented word sequence pair;

[0093] Concatenate the word vectors obtained through training to generate a word vector matrix.

[0094] For example, the multiple segmented words W obtained by splitting are: W = {w(1), w(2),..., w(n)}, and the corresponding parts of speech M are M = {m(1), m(2),..., m(n)}. Then, concatenate the multiple segmented words and corresponding parts of speech to construct a segmented word sequence pair X as: X = {(w(1), m(1)), (w(2), m(2)),..., (w(n), m(n))}.

[0095] Optionally, when using the Glove model to perform word vector training on the constructed segmented word sequence pair, establish a word vector dictionary, where each text vocabulary corresponds to a unique trained word vector. Optionally, the word vector dimension is set to 100; optionally, randomly initialize the word vectors of text vocabularies not appearing in the word vector dictionary.

[0096] Among them, concatenating the word vectors in the text vocabulary to generate a word vector matrix can be:

[0097]

[0098] Among them, w(1), w(2),..., w(i) represent the segmented words of the first interaction information; V(w(1)), V(w(2)),..., V(w(i)) represent the word vectors corresponding to the first interaction information; S ij represents the word vector matrix of the jth sentence concatenated by i word vectors; represents the concatenation of word vectors.

[0099] In a specific implementation manner of performing sentiment analysis on the first interaction information on the second playback interface to obtain the emotional characteristics corresponding to the first interaction information, after generating a word vector matrix according to multiple word segments and corresponding part-of-speech tags, a global semantic feature vector and a local semantic feature vector of the first interaction information are obtained according to the word vector matrix.

[0100] Optionally, obtaining the global semantic feature vector and the local semantic feature vector of the first interaction information according to the word vector matrix includes:

[0101] Inputting the word vector matrix into a TextCNN model to obtain the local semantic feature vector;

[0102] Inputting the word vector matrix into a BiGRU model to obtain global correlation information;

[0103] Performing correlation and / or logical analysis on the global correlation information by using a self-attention learning mechanism to obtain the global semantic feature vector.

[0104] By adopting this implementation manner, the word vector matrix is respectively used as the input of a BiGRU model and a TextCNN model introducing a self-attention (Self-Attention) learning mechanism. The BiGRU model introducing Self-Attention comprehensively extracts global features from two aspects of the syntactic structure of the text and the context information of the text to obtain a global semantic feature vector; the TextCNN model is used to extract the local semantic feature vector of the text;

[0105] Among them, the attention mechanism (Attention) mimics the internal process of biological observation behavior, that is, more attention is invested in important regions, a small amount of important information is selectively screened out and focused from a large amount of information, and at the same time, the interference of other information is suppressed. The self-attention mechanism (Self Attention) is a kind of attention mechanism, also known as the internal attention mechanism. Compared with the attention mechanism, the self-attention mechanism is better at capturing the deep internal logic between words in natural language expressions and the internal structure of sentences, and can generate a vector representation more consistent with the original text semantics, thereby improving the accuracy of sentiment analysis.

[0106] In one implementation manner, inputting the word vector matrix into a TextCNN model to obtain the local semantic feature vector, the specific implementation manner may be:

[0107] The word vector matrix S ijAs the input of the TextCNN model, optionally, the convolutional kernel sizes adopted by the TextCNN model are 3, 4, and 5 respectively, the number of convolutional kernels of each size is 32, the activation function is ReLU, the max pooling layer is used to reduce the dimension of the features extracted by the convolutional layer, and then the Concatenate() method in Keras is used to fuse the features extracted by different convolutional kernels to obtain the local semantic feature vector.

[0108] In this implementation, the local semantic feature vector o calculated according to the first interaction information at time t ijt The obtained representation can be:

[0109] o ijt = TextCNN(S ijt ).

[0110] In one implementation, optionally, the word vector matrix is input into the BiGRU model to obtain global correlation information. The specific implementation can be:

[0111] Taking the word vector matrix S ij as the input of the BiGRU model, setting the size of the hidden layer of the BiGRU model to 32, the activation function to Sigmoid, and inputting the word vector matrix S ij from two directions of the model respectively to extract the historical information and future information of the text; secondly, concatenating the outputs of the two directions to obtain the global correlation information h corresponding to time t ijt , where the calculation formula can be expressed as:

[0112] h ijt = BiGRU(S ijt ).

[0113] On the basis of obtaining the global correlation information h ijt in the above manner, a self-attention learning mechanism is further adopted to perform relevance and / or logical analysis on the global correlation information to obtain the global semantic feature vector.

[0114] Specifically, by introducing Self-Attention, the syntactic structure features of the text are learned, and the relevance and logic between words inside the sentence are deeply mined, so as to obtain a global semantic feature vector that is more in line with the original text semantics of the first interaction information; among them, the calculation formula of the global semantic feature vector is:

[0115] u ijt = tanh(W w h ijt + b w );

[0116]

[0117] att ijt = ∑ t a ijt h ijt 。

[0118] u ijt represents h ijt hidden layer unit; u w represents the context vector; a ijt represents the self-attention vector; att ijt represents the output vector of the self-attention mechanism, which is also the obtained global semantic feature vector; b w represents the bias value.

[0119] In the instant interaction method described in the embodiments of the present application, by performing sentiment analysis on the fused feature information after fusing the global semantic feature vector and the local semantic feature vector, the emotion feature corresponding to the first interaction information is obtained.

[0120] Specifically, by using the Concatenate function method to fuse the local semantic feature vectors of different fields extracted by the TextCNN model and the global semantic feature vectors extracted by the BiGRU model integrated with Self-Attention, the corresponding fused feature information d is obtained ijt , and the calculation formula is:

[0121]

[0122] After that, the Softmax function is used to perform text sentiment classification on the fused feature information d ijt to obtain the emotion feature emotion corresponding to the first interaction information i :

[0123] emotion i = Softmax(w i d ijt + b i );

[0124] where, w i represents the weight coefficient matrix from the fully connected Dense layer to the output layer; b i represents the corresponding bias; d ijt represents the output vector of the Dense layer at time t, which is also the fused feature information.

[0125] In the above manner, by obtaining the emotional characteristics of the first interaction information sent by the message-sending user, after obtaining the second interaction information replied by the message-replying user, the language text style can be converted according to the emotional characteristics, and the second interaction information can be converted into a reply message that conforms to the emotional characteristics, so as to realize the interaction content that conforms to the realistic emotional atmosphere.

[0126] In one implementation manner of the instant interaction method according to the embodiment of the present application, optionally, the method further includes:

[0127] Display the emotional information corresponding to the emotional characteristics in the form of bullet comments on the second playback interface.

[0128] In one implementation manner, optionally, the emotional information corresponding to the emotional characteristics is only displayed in the form of bullet comments on the second playback interface corresponding to the message-replying user.

[0129] In one implementation manner, optionally, the emotional information includes but is not limited to only being able to include irritability, anxiety, happiness, excitement, etc. Optionally, the emotional information corresponding to the emotional characteristics can be obtained from an emotional word library, and different emotional information corresponding to different emotional characteristics is recorded in the emotional word library.

[0130] Adopting this implementation manner, after obtaining the interaction content that conforms to the realistic emotional atmosphere, set emotional bullet comments on the second playback interface (conference small window) interaction interface (for example, digital human A says a paragraph to digital human B. Based on the emotional information of this paragraph said by digital human A, after the language style of this paragraph is converted, bullet comment content corresponding to the sentence after the language style conversion will appear on the screen at the same time. This bullet comment is only visible to digital human B, which is convenient for digital human B to understand the other party's emotions in a timely manner. Optionally, the bullet comment content is obtained from the emotional word library, and emotional vocabulary can be added to the emotional word library in advance. Based on the emotional analysis result of what digital human A said, emotional vocabulary consistent with the emotional analysis result is obtained from the emotional word library), making the emotions of the interaction partners visible, adding a human touch to the interaction process, and enhancing the immersive experience.

[0131] Illustrate by way of example, one implementation manner of the interface presented on the second playback interface (conference small window) can be as Figure 3As shown in the figure, the gray transparent area below the meeting small window interface is the meeting subtitle. By performing emotional analysis on the first interaction information input by digital human A: "There are too many arrangements today. I'm overwhelmed. Let's review the plan tomorrow.", the corresponding emotional characteristics are obtained. According to these emotional characteristics, for the second interaction information sentenceB1 replied by digital human B, the language style is converted to obtain the response message sentenceB2 that conforms to these emotional characteristics, so that the reply sentenceB2 of digital human B conforms to the context atmosphere of digital human A. At the same time, an emotional barrage that conforms to the emotional atmosphere of digital human A is generated. Optionally, the content of this emotional barrage is only visible to digital human B and not visible to digital human A. By adopting this implementation method, the emotions of digital human A are visualized through the emotional barrage to assist digital human B in having an "in-sync" interactive conversation and enhancing the immersive experience during the interaction process.

[0132] Optionally, the emotional barrage can be displayed by sliding on the meeting small window: taking Figure 3 as an example, if digital human A is on the left side of digital human B, the obtained emotional barrage that matches the context atmosphere of digital human A can slide from left to right to the side of digital human B. It should be noted that this sliding display method is only for illustrative purposes and is not limited thereto specifically.

[0133] The instant interaction method described in the embodiments of the present application realizes instant interaction between digital avatars by presenting a "small window interface" in the virtual metaverse meeting space, so as to enhance the immersive experience in the metaverse meeting scenario; in addition, by performing emotional analysis on the text information sent by the message sender and according to the emotional characteristics obtained from the analysis, the text information responded by the message answering user is converted into a response message that conforms to these emotional characteristics, making the interaction between digital avatars more emotional and adding a human touch to the interaction between digital humans in the virtual world.

[0134] One embodiment of the present application further provides an instant interaction device, as Figure 4 shown, including:

[0135] An interface establishment module 410, configured to present a second playback interface on the first playback interface used by the first user to play the metaverse meeting during the process of the metaverse meeting;

[0136] An information conversion module 420, configured to, during the process of the first user and the second user interacting information through the second playback interface, after obtaining the first interaction information on the second playback interface, convert the second interaction information in response to the first interaction information into a response message that conforms to the emotional characteristics corresponding to the first interaction information according to the emotional characteristics corresponding to the first interaction information;

[0137] A display module 430, configured to display the response message on the second playback interface.

[0138] In the instant interaction device described in the embodiments of the present application, by presenting a second playback interface on the first playback interface played in the metaverse conference and displaying the interaction information between the first user and the second user on the second playback interface, private communication between the first user and the second user is achieved; in addition, by performing sentiment analysis on the first interaction information and, according to the obtained emotional characteristics, converting the style of the second interaction information responded by the message answering user into a response information that conforms to the emotional characteristics, the interaction between the digital avatars becomes more emotional, so as to ensure the privacy, high-efficiency instantaneity and emotionality of the conversation between the digital humans corresponding to the first user and the second user, and ensure that the information interaction process is more in line with the digital human interaction scenario in the metaverse conference.

[0139] Optionally, in the instant interaction device, the information conversion module 420 is further configured to:

[0140] After obtaining the information sent by the message sending user among the first user and the second user on the second playback interface, obtain the first interaction information;

[0141] Perform sentiment analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information;

[0142] After obtaining the response information of the message answering user among the first user and the second user in response to the first interaction information, obtain the second interaction information.

[0143] Optionally, in the instant interaction device, the display module 430 is further configured to:

[0144] Display the emotional information corresponding to the emotional characteristics in the form of bullet screens on the second playback interface.

[0145] Optionally, in the instant interaction device, when the information conversion module 420 performs sentiment analysis on the first interaction information on the second playback interface to obtain the emotional characteristics corresponding to the first interaction information, it includes:

[0146] Perform word segmentation processing on the first interaction information to obtain a plurality of word segments and corresponding part-of-speech tags;

[0147] Generate a word vector matrix according to the plurality of word segments and corresponding part-of-speech tags;

[0148] According to the word vector matrix, obtain the global semantic feature vector and local semantic feature vector of the first interaction information;

[0149] Perform sentiment analysis on the fused feature information after fusing the global semantic feature vector and the local semantic feature vector to obtain the emotional characteristics corresponding to the first text information.

[0150] Optionally, for the instant interaction device described above, the information conversion module 420 obtains the global semantic feature vector and local semantic feature vector of the first text information according to the word vector matrix, including:

[0151] Input the word vector matrix into the TextCNN model to obtain the local semantic feature vector;

[0152] Input the word vector matrix into the BiGRU model to obtain the global correlation information;

[0153] Adopt a self-attention learning mechanism to perform correlation and / or logical analysis on the global correlation information to obtain the global semantic feature vector.

[0154] Optionally, for the instant interaction device described above, the interface establishment module 410 presents a second playback interface on the first playback interface used by the first user to play the metaverse conference, including:

[0155] During the metaverse conference, in response to a first request triggered by the first digital human corresponding to the first user on the first playback interface, establish the second playback interface on the first playback interface;

[0156] Among them, the interface establishment module 410 is further used for:

[0157] Determine the second user according to the second digital human selected by the first digital human on the first playback interface or the second playback interface;

[0158] Establish the second playback interface on the first playback interface used by the second user to play the metaverse conference.

[0159] Optionally, for the instant interaction device described above, the interface establishment module 410 is further used for:

[0160] When establishing the second playback interface, generate the number information and login password of the second playback interface;

[0161] Obtain a second request sent by a third digital human corresponding to a third user, where the second request is used to request to join the interaction between the first user and the second user, and the second request includes the number information and the login password;

[0162] When the verification of the number information and login password of the second playback interface is successful, establish the second playback interface on the first playback interface used by the third user to play the metaverse conference, and display the interaction information among the first user, the second user, and the third user on the second playback interface.

[0163] Optionally, for the instant interaction device described above, the display module 430 is further configured to:

[0164] Synchronously display the conference information played on the first playback interface on the second playback interface.

[0165] The instant interaction method and the instant interaction device in the embodiments of the present application are based on the same application concept. Since the principles for the method and the device to solve problems are similar, the implementations of the device and the method can be referred to each other, and the repeated parts will not be described again.

[0166] Another embodiment of the present application further provides a network device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, it implements the instant interaction method described in any one of the above.

[0167] Among them, for the program running on the processor of the network device, the specific implementation of executing the instant interaction method can refer to the detailed description when the instant interaction method is applied to the network device, and will not be described repeatedly here.

[0168] In addition, a specific embodiment of the present application further provides a readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps in the instant interaction method described in any one of the above.

[0169] Specifically, the readable storage medium is applied to the above-mentioned network device. When applied to the network device, the execution steps corresponding to the instant interaction method are as described in detail above, and will not be described again here.

[0170] Another embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they implement the steps in the instant interaction method described in any one of the above, or implement the steps in the instant interaction method described in any one of the above.

[0171] Optionally, the embodiments of the present application may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0172] For the computer program product in the embodiments of the present application, the computer instructions included implement each process of the method embodiment as shown above when executed by a processor, and can achieve the same technical effects. To avoid repetition, it will not be described again here.

[0173] In several embodiments provided by the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0174] In addition, the functional units in each embodiment of the present application can be integrated in a processing unit, or each unit can be physically included separately, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0175] The above-mentioned integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above-mentioned software functional units are stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute some steps of the transceiver methods described in each embodiment of the present application. The foregoing storage medium includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM for short), random access memories (RAM for short), magnetic disks, or optical discs.

[0176] The above are the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle described in the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An instant interaction method, characterized in that, Including: During the Metaverse conference, a second playback interface is presented on the first playback interface used by the first user to play the Metaverse conference; During the process of the first user and the second user interacting information through the second playback interface, after obtaining the first interaction information on the second playback interface, according to the emotional characteristics corresponding to the first interaction information, the second interaction information in response to the first interaction information is converted into a response message that conforms to the emotional characteristics; The response message is displayed on the second playback interface.

2. The instant interaction method according to claim 1, wherein The method further includes: After obtaining the information sent by the message sending user among the first user and the second user on the second playback interface, the first interaction information is obtained; Performing sentiment analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information; After obtaining the response information of the message answering user among the first user and the second user in response to the first interaction information, the second interaction information is obtained.

3. The instant interaction method according to claim 1 or 2, characterized in that, The method further includes: Displaying the emotion information corresponding to the emotional characteristics in the form of bullet comments on the second playback interface.

4. The instant interaction method according to claim 2, wherein Performing sentiment analysis on the first interaction information to obtain the emotional characteristics corresponding to the first interaction information, including: Performing word segmentation processing on the first interaction information to obtain multiple word segments and corresponding parts of speech; Generating a word vector matrix according to the multiple word segments and corresponding parts of speech; Obtaining the global semantic feature vector and local semantic feature vector of the first interaction information according to the word vector matrix; Performing sentiment analysis on the fused feature information after fusing the global semantic feature vector and the local semantic feature vector to obtain the emotional characteristics corresponding to the first interaction information.

5. The instant interaction method according to claim 4, wherein Obtaining the global semantic feature vector and local semantic feature vector of the first interaction information according to the word vector matrix, including: Inputting the word vector matrix into a TextCNN model to obtain the local semantic feature vector; Inputting the word vector matrix into a BiGRU model to obtain global correlation information; Using a self-attention learning mechanism to perform correlation and / or logical analysis on the global correlation information to obtain the global semantic feature vector.

6. The instant interaction method according to claim 1, characterized in that, During the Metaverse conference, presenting a second playback interface on the first playback interface used by the first user to play the Metaverse conference, including: During the Metaverse conference, in response to a first request triggered by the first digital human corresponding to the first user on the first playback interface, establishing the second playback interface on the first playback interface; Wherein, the method further includes: Determining the second user according to the second digital human selected by the first digital human on the first playback interface or the second playback interface; Establishing the second playback interface on the first playback interface used by the second user to play the Metaverse conference.

7. The instant interaction method according to claim 1 or 6, characterized in that, The method further includes: When establishing the second playback interface, generating the number information and login password of the second playback interface; Obtain a second request sent by the third digital human corresponding to the third user, where the second request is used to request to join the interaction between the first user and the second user, and the second request includes the number information and the login password; When the number information and the login password of the second playback interface are successfully verified, establish the second playback interface on the first playback interface used by the third user to play the metaverse meeting, and display the interaction information among the first user, the second user, and the third user on the second playback interface.

8. An instant interaction device, characterized in that, Comprising: An interface establishment module, configured to present a second playback interface on the first playback interface used by a first user to play the metaverse meeting during the metaverse meeting; An information conversion module, configured to, during the process of the first user and the second user interacting information through the second playback interface, after obtaining first interaction information on the second playback interface, convert second interaction information in response to the first interaction information into a response message conforming to the emotional characteristics according to the emotional characteristics corresponding to the first interaction information; A display module, configured to display the response message on the second playback interface.

9. A network device, characterized in that, Comprising a processor, a memory, and a program stored on the memory and executable on the processor, where when the program is executed by the processor, it implements the instant interaction method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, A program is stored on the readable storage medium, and when the program is executed by a processor, it implements the steps in the instant interaction method according to any one of claims 1 to 7.

Images