Call method and apparatus, and electronic device

Abstract

The present disclosure provides a call method and apparatus, and an electronic device. The method comprises: receiving a call request initiated by a first user to a second user, wherein the first user and the second user are both pre-registered with a real identity and a virtual identity; in response to the call request, determining call identities of the first user and the second user, wherein the call identities are real identities or virtual identities; on the basis of the call identity determination result of the first user and the second user, carrying out Session Description Protocol (SDP) negotiation corresponding to the call identity determination result; and when the SDP negotiation is completed, establishing a multimedia call between the first user and the second user.

Description

Call method, device and electronic equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese Patent Application No. 202311064411.2 filed in China on August 23, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of communication technology, and in particular to a communication method, device, and electronic device. Background Art

[0004] With the continuous development of Internet communication technology, more and more business scenarios have multimedia call functions for audio and video. With the development of metaverse technology, more and more business scenarios are applying metaverse technology to multimedia calls to improve the call experience.

[0005] At present, the commonly used calling method is that the user initiates a call request through his real identity and calls the other party's real identity. However, the goal of user calls in the metaverse environment (scenario) is to achieve the gradual integration of the real world and the virtual world. The above commonly used calling method cannot meet the user's virtual and real twin call needs in the metaverse environment, resulting in a poor calling experience in the metaverse environment.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a calling method, device, and electronic device to solve the problem of poor calling experience in the prior art.

[0008] To solve the above technical problems, the present disclosure is implemented as follows:

[0009] In a first aspect, an embodiment of the present disclosure provides a call method, applied to a Metaverse server, the method comprising:

[0010] Receiving a call request initiated by a first user to a second user, wherein both the first user and the second user have pre-registered real identities and virtual identities;

[0011] In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities;

[0012] Based on the call identity determination results of the first user and the second user, performing a Session Description Protocol (SDP) negotiation corresponding to the call identity determination results;

[0013] When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

[0014] In a second aspect, an embodiment of the present disclosure provides a call method, applied to a first user terminal where a first user is located, the method comprising:

[0015] A call request is sent to a Metaverse server, where the call request is a call request initiated by the first user to the second user, wherein the first user and the second user both have pre-registered real identities and virtual identities, and the call request is used to request the Metaverse server to perform a Session Description Protocol (SDP) negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

[0016] In a third aspect, an embodiment of the present disclosure provides a communication device, applied to a Metaverse server, comprising:

[0017] A first receiving module is configured to receive a call request initiated by a first user to a second user, wherein both the first user and the second user are pre-registered with a real identity and a virtual identity;

[0018] a call identity determination module, configured to determine, in response to the call request, the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities;

[0019] a negotiation module, configured to perform a session description protocol (SDP) negotiation corresponding to a call identity determination result based on a call identity determination result of the first user and the second user;

[0020] The call establishing module is used to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

[0021] In a fourth aspect, an embodiment of the present disclosure provides a communication device, applied to a first user terminal where a first user is located, the device comprising:

[0022] A first sending module, configured to send a call request to a Metaverse server;

[0023] Among them, the call request is a call request initiated by the first user to the second user. The first user and the second user have both pre-registered real identities and virtual identities. The call request is used to request the metaverse server to conduct a session description protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

[0024] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including a transceiver and a processor,

[0025] The processor is configured to:

[0026] receiving, by the transceiver, a call request initiated by a first user to a second user, wherein both the first user and the second user are pre-registered with a real identity and a virtual identity;

[0027] In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities;

[0028] Based on the call identity determination results of the first user and the second user, performing a Session Description Protocol (SDP) negotiation corresponding to the call identity determination results;

[0029] When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

[0030] In a sixth aspect, an embodiment of the present disclosure provides an electronic device, including a transceiver and a processor,

[0031] The processor is configured to send a call request to a Metaverse server via the transceiver;

[0032] Among them, the call request is a call request initiated by the first user to the second user. The first user and the second user have both pre-registered real identities and virtual identities. The call request is used to request the metaverse server to conduct a session description protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

[0033] In the seventh aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the call method described in the first aspect are implemented, or the steps of the call method described in the second aspect are implemented.

[0034] In an eighth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the call method described in the first aspect are implemented, or the steps of the call method described in the second aspect are implemented.

[0035] In the call method of this embodiment, both users can pre-register their real identities and virtual identities. That is, both the first user and the second user have real identities and virtual identities. When the Metaverse server receives a call request from the first user to the second user, it can determine the call identities of the first and second users. The call identity of the first user can be either a real identity or a virtual identity, and the call identity of the second user can be either a real identity or a virtual identity. Subsequently, a Session Description Protocol (SDP) negotiation corresponding to the call identity determination results is performed based on the call identity determination results of the first and second users. It can be understood that the call identities of the first and second users are different, and different SDP negotiation processes can be performed. When the negotiation is completed, a multimedia call between the first and second users can be established. That is, in this embodiment, regardless of whether the first user is a real identity or a virtual identity, and whether the second user is a real identity or a virtual identity, an SDP negotiation corresponding to the call identity judgment result can be performed based on the call identity judgment result of the first user and the second user to establish a multimedia call between the first user and the second user. That is, the first user can not only talk to the second user with a real identity or a virtual identity through a virtual identity, but also talk to the second user with a real identity or a virtual identity through a real identity. In this way, the user's virtual and real twin call needs in the metaverse environment can be met, and the user's call experience in the metaverse environment can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] FIG1 is a flowchart of a calling method according to an embodiment of the present disclosure;

[0038] FIG2 is a second flowchart of a communication method provided by an embodiment of the present disclosure;

[0039] FIG3 is a third flowchart of a communication method provided by an embodiment of the present disclosure;

[0040] FIG4 is a fourth flowchart of a communication method provided by an embodiment of the present disclosure;

[0041] FIG5 is a fifth flowchart of a communication method provided by an embodiment of the present disclosure;

[0042] FIG6 is a sixth flowchart of a communication method provided by an embodiment of the present disclosure;

[0043] FIG7 is a seventh flowchart of a communication method provided by an embodiment of the present disclosure;

[0044] FIG8 is an eighth flowchart of a communication method provided by an embodiment of the present disclosure;

[0045] FIG9 is a ninth flowchart of a communication method provided by an embodiment of the present disclosure;

[0046] FIG10 is a flowchart of a call method according to an embodiment of the present disclosure;

[0047] FIG11 is a schematic structural diagram of a communication device provided in an embodiment of the present disclosure;

[0048] FIG12 is a schematic structural diagram of a communication device provided in an embodiment of the present disclosure;

[0049] FIG13 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

[0050] FIG14 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0052] Refer to Figure 1, which is a flow chart of a call method provided by an embodiment of the present disclosure, which is applied to a Metaverse server and can be applied to Metaverse scenarios. As shown in Figure 1, the call method provided by this embodiment includes the following steps:

[0053] Step 101: receiving a call request initiated by a first user to a second user, wherein both the first user and the second user have pre-registered real identities and virtual identities;

[0054] Step 102: In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities;

[0055] Step 103: Based on the call identity determination results of the first user and the second user, a Session Description Protocol (SDP) negotiation corresponding to the call identity determination results is performed;

[0056] Step 104: When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

[0057] It can be understood that in this embodiment, users can pre-register in the Metaverse server and can register real identities and virtual identities. Users can pass through with their real identities or talk with their virtual identities.

[0058] It should be noted that in the Metaverse environment, when a user needs to make a call, he or she must first log in to the Metaverse server. During the login process, the user can choose to log in with a virtual identity or a real identity. In addition, the user can log in with a real identity and make a call. At this time, the user's call identity is the real identity. The Metaverse server can record the user's call identity through the call identity identifier based on the user's login request. At this time, the recorded call identity identifier can indicate that the user's call identity is the real identity. After logging off with the real identity, the user can continue to make a call with a virtual identity. At this time, the recorded call identity identifier can indicate that the user's call identity is the virtual identity. It should be understood that the latest call identity of each user can be recorded in the Metaverse server. After receiving the call request from the first user to the second user, the current call identity of the first user and the second user can be determined.

[0059] In addition, in the SDP negotiation process of the embodiment of the present disclosure, SDP negotiation is performed through the call identity judgment results of the first user and the second user. It can be understood that the SDP negotiation process is related to the call identity judgment results of the first user and the second user. After the negotiation is completed, a multimedia call is established between the first user and the second user.

[0060] In the call method of this embodiment, each user can pre-register a real identity and a virtual identity. That is, both the first user and the second user have a real identity and a virtual identity. When the Metaverse server receives a call request from the first user to the second user, it can determine the call identities of the first and second users. The call identity of the first user can be either a real identity or a virtual identity, and the call identity of the second user can be either a real identity or a virtual identity. Subsequently, based on the call identity determination results of the first and second users, a Session Description Protocol (SDP) negotiation corresponding to the call identity determination results is performed. It can be understood that the different call identities of the first and second users can lead to different SDP negotiation processes. Upon completion of the negotiation, a multimedia call between the first and second users can be established. That is, in this embodiment, regardless of whether the first user is a real identity or a virtual identity, and whether the second user is a real identity or a virtual identity, SDP negotiation corresponding to the call identity judgment result can be performed based on the call identity judgment result of the first user and the second user to establish a multimedia call between the first user and the second user, that is, the first user can not only talk to the second user with a real identity or a virtual identity through a virtual identity, but also talk to the second user with a real identity or a virtual identity through a real identity. In this way, the user's virtual and real twin call needs in the metaverse environment can be met, and the user's call experience in the metaverse environment can be improved.

[0061] In one embodiment, before receiving a call request initiated by a first user to a second user, at least one of the following is further included:

[0062] When a first user logs in with a real identity and the first user's virtual identity is in a call, the first user's real identity replaces the first user's virtual identity to conduct the call, wherein the call request is a request initiated by the first user with the real identity;

[0063] When the first user logs in with a real identity and the first user's virtual identity is not in a call, the first user's virtual identity call function is turned off, and the first user is determined to be making a call with a real identity, wherein the call request is a request initiated by the first user with the real identity.

[0064] In this embodiment, a process for users to go online with their real identities is provided. For example, when user A logs in with his real identity, it can be determined whether the user's virtual identity V(A) is making a call. If so, the real identity R(A) is used instead of the virtual identity V(A) to make the call. If not, the call function of the virtual identity V(A) is turned off, and subsequent calls are made with the real identity R(A) to improve the user's call flexibility.

[0065] In one embodiment, after establishing the multimedia call between the first user and the second user, the method further includes at least one of the following:

[0066] When the real identity of the first user goes offline and the real identity of the first user is in a call before the first user goes offline, the first user's virtual identity replaces the real identity of the first user to conduct the call;

[0067] When the real identity of the first user goes offline and the real identity of the first user is not in a call before the first user goes offline, the real identity call function of the first user is disabled, and the first user is determined to make a call with a virtual identity.

[0068] In this embodiment, a process is provided for a user to log off using their real identity. For example, if user A logs off using their real identity, the process determines whether their real identity R(A) was previously active. If so, the virtual identity V(A) replaces the real identity R(A) for the call. If not, the virtual identity V(A) call function is activated, and subsequent calls are made using the virtual identity V(A) to increase the user's call flexibility. In addition, even if the user's real identity is offline, they can still make calls using the virtual identity, which can meet the twin call needs in the metaverse environment and improve the call experience.

[0069] In one embodiment, based on the call identity determination results of the first user and the second user, a Session Description Protocol (SDP) negotiation is performed, including at least one of the following:

[0070] When the call identities of the first user and the second user are both real identities, SDP negotiation is performed based on the first real audio media information of the first user, the second real audio media information of the second user, the first digital human image of the first user, and the second digital human image of the second user;

[0071] In a case where the call identity of at least one of the first user and the second user is a virtual identity, SDP negotiation is performed based on at least artificial intelligence (AI) audio and video media information of the at least one user.

[0072] That is, in this embodiment, when the call identities of the first user and the second user are both real identities, the first real audio media information of the first user, the second real audio media information of the second user, the first digital human image of the first user, and the second digital human image of the second user are used to perform SDP negotiation. When the call identity of at least one of the first user and the second user is a virtual identity (the call identity of the first user is a virtual identity, or the call identity of the second user is a virtual identity, or the call identity of the first user and the call identity of the second user are both virtual identities), the AI ​​audio and video media information of at least one user is used to perform SDP negotiation. It should be understood that for the first user and the second user, different SDP negotiation processes can be adopted under different call identity judgment results to improve the flexibility of SDP negotiation.

[0073] In one embodiment, the call request carries the first real audio media information;

[0074] Performing SDP negotiation based on first real audio media information of a first user, second real audio media information of a second user, a first digital human image of the first user, and a second digital human image of the second user, including:

[0075] According to the first real audio media information, driving the first digital human image corresponding to the first user to move, and obtaining a first digital human video;

[0076] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0077] Sending a first message to the second user, where the first message includes the first audio and video media information and the call identity of the first user;

[0078] receiving a first reply message returned by the second user, where the first reply message includes second real audio media information of the second user;

[0079] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0080] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0081] When the second message is sent to the first user, it is determined that the SDP negotiation is completed. The second message includes the second audio and video media information and the call identity of the second user.

[0082] In one embodiment, SDP negotiation is performed based on at least one user's AI audio and video media information, including:

[0083] When at least one user is the first user, a third message is sent to the second user, where the third message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information, and the first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0084] receiving a second reply message returned by the second user, where the second reply message includes second real audio media information of the second user;

[0085] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0086] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0087] When the fourth message is sent to the first user, it is determined that the SDP negotiation is completed. The fourth message includes the second audio and video media information and the call identity of the second user.

[0088] It can be understood that in this embodiment, when at least one user is the first user, the AI ​​audio and video media information of at least one user is the first AI audio and video media information of the first user, and the second user is a real identity. SDP negotiation can be performed based on the second real audio media information of the second user and the AI ​​audio and video media information of at least one user. That is, during the negotiation process, not only the AI ​​audio and video media information of the first user with the virtual identity is used, but also the second real audio media information of the second user is used to facilitate the establishment of a multimedia call between the first user with the virtual identity and the second user with the real identity.

[0089] In one embodiment, SDP negotiation is performed based on at least one user's AI audio and video media information, including:

[0090] In the case where at least one user is the second user, driving the first digital human image corresponding to the first user to move according to the first real audio media information carried in the call request to obtain the first digital human video;

[0091] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0092] Sending a fifth message to the second user, where the fifth message includes the first audio and video media information and the call identity of the first user;

[0093] Receive a third reply message returned by the second user, the third reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0094] When the sixth message is sent to the first user, it is determined that the SDP negotiation is completed. The sixth message includes the second AI audio and video media information and the call identity of the second user.

[0095] It can be understood that in this embodiment, when at least one user is the second user, the AI ​​audio and video media information of at least one user is the second AI audio and video media information of the second user, and the first user is a real identity. SDP negotiation can be performed based on the first real audio media information of the first user and the AI ​​audio and video media information of at least one user. That is, during the negotiation process, not only the AI ​​audio and video media information of the second user with the virtual identity is used, but also the first real audio media information of the first user is used to facilitate the establishment of a multimedia call between the second user with the virtual identity and the first user with the real identity.

[0096] In one embodiment, SDP negotiation is performed based on at least one user's AI audio and video media information, including:

[0097] When at least one user is a first user and a second user, a seventh message is sent to the second user. The seventh message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information. The first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0098] Receive a fourth reply message returned by the second user, the fourth reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0099] When the eighth message is sent to the first user, it is determined that the SDP negotiation is completed. The eighth message includes the second AI audio and video media information and the call identity of the second user.

[0100] It can be understood that in this embodiment, when at least one user includes a first user and a second user, the AI ​​audio and video media information of at least one user includes the first AI audio and video media information of the first user and the second AI audio and video media information of the second user. The first user and the second user are both virtual identities. SDP negotiation can be performed based on the first AI audio and video media information of the first user and the second AI audio and video media information of the second user to facilitate establishing a multimedia call between the first user with the virtual identity and the second user.

[0101] In one embodiment, after the SDP negotiation is completed, establishing a multimedia call between the first user and the second user includes:

[0102] When the SDP negotiation is completed, receiving a first confirmation message sent by the second user;

[0103] Sending a first confirmation message to the first user;

[0104] receiving a second confirmation message sent by the first user after receiving the first confirmation message;

[0105] When the second confirmation message is sent to the second user, the multimedia call between the first user and the second user is established.

[0106] In one embodiment, after establishing the multimedia call between the first user and the second user, the method further includes:

[0107] When a first user logs in with a real identity and conducts a multimedia call with a second user with a virtual identity, receiving a first target call request message sent by the first user with the real identity, the first target call request message including third real audio media information of the first user;

[0108] In response to the first target call request message, the current call is divided into a sub-call segment between the first user using the virtual identity to the Metaverse server and a sub-call segment between the Metaverse server and the second user;

[0109] Superimposing the third real audio media information onto the digital human video of the first user to generate third audio and video media information;

[0110] Sending a second target call request message to the second user, where the second target call request message includes the third audio and video media information and the call identity of the first user;

[0111] Receiving a first response message sent by the second user, where the first response message includes real audio media information or AI audio and video media information of the second user;

[0112] Sending a target message to the first user, where the target message includes the second user's real audio media information or AI audio and video media information, and the second user's call identity;

[0113] Cut off the sub-session segment of the first user using the virtual identity to the Metaverse server, and establish a sub-session segment of the first user using the real identity to the Metaverse server.

[0114] It should be noted that if the second user calls with a real identity, the above-mentioned first response message includes the second user's real audio media information (which can be understood as the second user's fourth real audio media information); if the second user calls with a virtual identity, the above-mentioned first response message includes the second user's AI audio and video media information (which can be understood as the second user's third AI audio and video media information). The second user's AI audio and video media information here can be obtained by superimposing the second user's current AI audio on the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement by the second user's current AI audio.

[0115] Referring to FIG. 2 , FIG. 2 is a flowchart of a calling method provided by an embodiment of the present disclosure, which is executed by a first user terminal where a first user is located. As shown in FIG. 2 , the calling method provided by this embodiment includes the following steps:

[0116] Step 201: Send a call request to the Metaverse server. The call request is a call request initiated by the first user to the second user, wherein the first user and the second user both have pre-registered real identities and virtual identities. The call request is used to request the Metaverse server to perform a session description protocol SDP negotiation corresponding to the call identity judgment results based on the call identity judgment results of the first user and the second user, and establish a multimedia call between the first user and the second user after the SDP negotiation is completed.

[0117] In one embodiment, before sending the call request to the Metaverse server, the process further includes:

[0118] A login request is sent to the Metaverse server with the real identity of the first user. The login request is used to request the Metaverse server to replace the first user's virtual identity with the real identity of the first user when the first user's virtual identity is in a call to make a call, or to turn off the first user's virtual identity call function when the first user's virtual identity is not in a call, to determine that the first user is making a call with his real identity.

[0119] In one embodiment, after sending the call request to the Metaverse server, the method further includes:

[0120] Send an offline request to the Metaverse server. The offline request is used to request the Metaverse server to replace the first user's real identity with the first user's virtual identity to make a call if the first user's real identity is in a call before receiving the offline request, or to turn off the first user's real identity call function if the first user's real identity is not in a call before receiving the offline request, and determine that the first user is making a call with a virtual identity.

[0121] The process of the above method is described in detail below using a specific embodiment. For example, the first user is user A and the second user is user B. The call identity identifier is used to identify the call identity. For example, a user's call identity identifier is 1, indicating that the user's call identity is a real identity, and a user's call identity identifier is 0, indicating that the user's call identity is a virtual identity. The call identity identifier is an audio and video separation identifier.

[0122] First, let’s explain the relevant definitions involved:

[0123] Multimedia Telephony: MMTel (Multi Media Telephony) is a multimedia telephony service standard based on the Internet Protocol (IP) Multimedia Subsystem (IMS) jointly developed by the European Telecommunications Standards Institute (ETSI) / Telecommunications and Internet Converged Services and Protocols for Advanced Networking (TISPAN) and the 3rd Generation Partnership Project (3GPP). It is a global standard that supports telecom operators in evolving traditional circuit-switched services into real-time IP-based multimedia telephony services, including voice and video calls, messaging, and video image sharing. It also supports both fixed and mobile broadband access.

[0124] Digital Human: A digitized humanoid created using digital technology that closely resembles a human. The digital human mentioned in this patent can be understood as an avatar representing a real-world user in the virtual world. This avatar can, under the user's control, perform activities in the virtual world on their behalf, including making calls and performing related actions.

[0125] The current implementation of this service is that real-world users can communicate with their digital avatars when they are online, but cannot be contacted after they go offline. This service experience does not meet the needs of users in the metaverse. The goal of user communication in the metaverse environment is to gradually integrate the real and virtual worlds, ultimately realizing digital twin communication in the metaverse.

[0126] The main idea of ​​this disclosure is: two (or more) users communicate in the metaverse space, and each user has two identities: a real identity (R) and a virtual identity (V). For user 1 and user 2, there are four call modes: R1-R2, R1-V2, V1-R1, and V1-V2. This disclosure realizes real-time twin calls without the two parties being aware of each other's virtuality or reality.

[0127] Figure 3 shows a flowchart of the overall implementation of an embodiment of the present disclosure, which provides a method for two users to communicate in a metaverse scenario. The method includes: a user initiates registration, registering both a real and a virtual identity; one user initiates a call request to another user; the network determines whether the two users are communicating with their real identities. If both users are communicating with their real identities, the network uses the real user's audio to drive the relevant lip movements and body movements of the digital human, and synchronizes the digital human's video image with the audio to form a call SDP; if both users are virtual identities or one is a virtual identity, the virtual user uses their own digital human video image and AI audio to form a call SDP; and then establishes an audio and video call between the two users.

[0128] To implement the above-mentioned call method, the disclosed embodiment may include: a user call identity selection process, a call process between two users in the metaverse, and a user real / virtual identity switching process during a call. As shown in Figures 4 and 5, the specific process methods are as follows:

[0129] The user call identity selection process disclosed in this disclosure is described as follows:

[0130] Figure 4 shows the real-identity user online process:

[0131] 1) User A logs in with real identity;

[0132] 2) Determine whether the virtual identity V(A) is in a call;

[0133] 3) If yes, the call is made with the real identity R(A) instead of the virtual identity V(A);

[0134] 4) If not, the call function of the virtual identity V(A) is closed, and subsequent calls are made with the real identity R(A).

[0135] Figure 5 shows the offline process for real-identity users:

[0136] 1) User A logs off with his real identity;

[0137] 2) Determine whether the real identity R(A) was in a call before;

[0138] 3) If yes, the call is made with the virtual identity V(A) instead of the real identity R(A);

[0139] 4) If not, the virtual identity V(A) call function is activated and subsequent calls are made with the virtual identity V(A).

[0140] FIG6 is a business process of the present disclosure in which both parties in a call are real people. The process is described as follows:

[0141] 1) After user A logs in to the service with his real identity, he sends a request (INVITE) message (also called a call request, carrying user A's real audio and media information) to the Metaverse server. The audio information drives the digital human's movements. User A's real audio and media information is superimposed on the digital human's video to construct the SDP. The SDP attribute information: audio and video separation. In addition, the Metaverse server sends a response message (100Trying) to user A.

[0142] 2) The Metaverse server sends the first message (carrying user A's audio and video media information (generated by superimposing user A's real audio media information onto user A's digital human video), audio and video separation flag = 1) to user B (who logs in with their real identity);

[0143] 3) User B returns the first reply message (e.g., 18X message, containing user B's real audio media information) to the Metaverse server. The audio information drives the digital human's movements. User B's real audio information is superimposed on the digital human video to construct the SDP. The SDP attribute information is: audio and video separation.

[0144] 4) The Metaverse server sends the second message (carrying user B's audio and video media information (generated by superimposing user B's real audio media information onto user A's digital human video, with audio and video separation flag = 1) to user A, completing the SDP negotiation.

[0145] 5) User B connects the session and sends a success response to the Metaverse server (corresponding to the first confirmation message mentioned above, for example, 200OK). The Metaverse server forwards it to user A, and user A replies with a second confirmation message (for example, ACK). The audio and video call media stream between user A and user B is established.

[0146] Users can determine whether the other party is a digital person controlled by a real identity or a digital person with a virtual identity based on the audio and video separation identifier.

[0147] FIG7 is a business process of a real person calling a virtual person in the present disclosure, and the process is described as follows:

[0148] 1) After user A logs in to the service with his real identity, he sends a request (INVITE) message (carrying user A's real audio media information) to the Metaverse server. The audio information drives the digital human's movements. User A's real audio media information is superimposed on user A's digital human video to construct the SDP. The SDP attribute information: audio and video separation. In addition, the Metaverse server sends a response message (100Trying) to user A.

[0149] 2) The Metaverse server sends a fifth message (carrying user A's audio and video media information (generated by superimposing user A's real audio media information onto user A's digital human video), audio and video separation flag = 1) to user B (virtual identity login);

[0150] 3) User B replies with a third reply message (carrying user B's AI audio and video media information (obtained by superimposing user B's AI audio onto user B's digital human video)) to the Metaverse server;

[0151] 4) The Metaverse server sends a sixth message (carrying user B's AI audio and video media information, with the audio and video separation flag = 0) to user A, completing the SDP negotiation.

[0152] 5) User B connects the session and sends a success response (200OK) to the Metaverse server. The Metaverse server forwards it to User A. User A replies with a second confirmation message (ACK), and the audio and video call media stream between User A and User B is established.

[0153] FIG8 is a business process of a virtual person calling a real person in the present disclosure, and the process is described as follows:

[0154] 1) Virtual identity user A sends a request (INVITE) message (carrying user A's AI audio and video media information) to the Metaverse server; in addition, the Metaverse server sends a response message (100Trying) to user A;

[0155] 2) The Metaverse server sends the third message (carrying user A's AI audio and video media information, audio and video separation flag = 0) to user B (real identity login);

[0156] 3) User B sends a second reply message (carrying user B's real audio media information) to the Metaverse server. The audio information drives the digital human's movements. User B's real audio is superimposed on user B's digital human video to construct the SDP. The SDP attribute information: audio and video separation;

[0157] 4) The Metaverse server sends a fourth message (carrying user B's audio and video media information (user B's real audio media information superimposed on user B's digital human video), audio and video separation flag = 1) to user A, completing the SDP negotiation;

[0158] 5) User B connects to the session and sends a success response (200OK) to the Metaverse server. The Metaverse server forwards it to User A, and User A replies with a confirmation message (ACK). The audio and video call media stream between User A and User B is established.

[0159] FIG9 is a business process of a virtual person calling a virtual person in the present disclosure, and the process is described as follows:

[0160] 1) Virtual identity user A sends a request (INVITE) message (carrying user A's AI audio and video media information) to the Metaverse server; in addition, the Metaverse server sends a response message (100Trying) to user A;

[0161] 2) The Metaverse server sends a seventh message (carrying user A's AI audio and video media information (obtained by superimposing user A's AI audio onto user A's digital human video), with audio and video separation flag = 0) to virtual identity user B;

[0162] 3) User B replies with a fourth reply message (carrying user B's AI audio and video media information (obtained by superimposing user B's AI audio onto user B's digital human video)) to the Metaverse server;

[0163] 4) The Metaverse server sends the eighth message (carrying user B's AI audio and video media information, audio and video separation flag = 0) to user A, completing the SDP negotiation;

[0164] 5) User B connects the session and sends a success response (200OK) to the Metaverse server. The Metaverse server forwards it to User A. User A replies with a second confirmation message (ACK), and the audio and video call media stream between User A and User B is established.

[0165] In the embodiment of the present disclosure, the process of switching between a user's real and virtual identities during a call is as follows:

[0166] FIG10 is a business process of a real person taking over a call from a virtual person in the present disclosure. The process is described as follows:

[0167] 1) User A uses a virtual identity to conduct an audio or video call with user B. User A logs in with their real identity and detects that their virtual identity is in the call. The real identity sends a re-request (ReINVITE, corresponding to the first target call request message mentioned above) message (carrying user A's real audio media information, which can be understood as user A's third real audio media information) to the Metaverse server. In addition, the Metaverse server returns a response message to user A.

[0168] 2) The Metaverse server divides the existing session into a media segment from the virtual user to the server and a media segment from the server to user B. It also copies user A's digital human image and overlays the digital human video SDP, with the SDP attribute information indicating audio and video separation. The Metaverse server sends a second target call request message (carrying user A's real audio and video media information (which can be understood as user A's third audio and video media information, generated by superimposing user A's real audio media information onto user A's digital human video), with the audio and video separation flag = 1) to user B.

[0169] 3) User B replies with a first response message (carrying user B's real audio media information or AI audio and video media information) to the Metaverse server, and the Metaverse server sends a target message to user A (carrying user B's real audio media information or AI audio and video media information, as well as user B's audio and video separation identifier);

[0170] 4) The original session between user A (virtual user) and the server is disconnected, and a session between user A and the server is established. User A sends an acknowledgment message (ACK) to user B, and user A conducts an audio and video call with user B using their real identity.

[0171] The process for a real-identity user to go offline or transfer the conversation to a virtual identity during a call is similar to the above, with the difference being: in step 2), there is no need to copy the digital human image. The Metaverse server can directly use AI audio and video media information to send a ReINVITE message (carrying AI audio and video media information, audio and video separation flag = 0) to user B.

[0172] In summary, the calling method of the embodiment of the present disclosure provides a calling method for scenarios where two (or more) users communicate in the metaverse space using real and virtual identities, and provides a process for users to select user calling identities in the metaverse space, including processes for real people going online and real people going offline; and in the user's call in the metaverse, a business process method is provided in which both parties of the call are real people, a process method for real people calling virtual people, a process method for virtual people calling real people, and a process method for virtual people calling virtual people. In addition, in the process of switching between the real and virtual identities of users during a call, a business process method is provided in which a real person takes over a call from a virtual person, and a method in which a real identity user goes offline or transfers a conversation to a virtual identity during a call. Through the digital twin calling method in the metaverse scenario of the present disclosure, the real world and the virtual world are seamlessly integrated.

[0173] As shown in FIG. 11 , FIG. 11 is a communication device 1100 provided in an embodiment of the present disclosure, which is applied to a Metaverse server. The device 1100 includes:

[0174] A first receiving module 1101 is configured to receive a call request initiated by a first user to a second user, wherein both the first user and the second user have pre-registered real identities and virtual identities;

[0175] A call identity determination module 1102 is configured to determine the call identities of the first user and the second user in response to the call request, wherein the call identities are real identities or virtual identities;

[0176] The negotiation module 1103 is configured to perform a session description protocol SDP negotiation corresponding to the call identity determination result based on the call identity determination result of the first user and the second user;

[0177] The call establishing module 1104 is configured to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

[0178] In one embodiment, the communication device 1100 further includes at least one of the following:

[0179] A first replacement module, configured to replace the first user's virtual identity with the first user's real identity before the first receiving module 1101 receives a call request initiated by the first user to the second user, when the first user logs in with the real identity and the first user's virtual identity is in a call, so as to conduct the call, wherein the call request is a request initiated by the first user with the real identity;

[0180] The second replacement module is used to turn off the virtual identity call function of the first user and determine that the first user is making a call with his real identity before the first receiving module 1101 receives the call request initiated by the first user to the second user, when the first user logs in with his real identity and the first user's virtual identity is not in a call, wherein the call request is a request initiated by the first user with his real identity.

[0181] In one embodiment, the communication device 1100 further includes at least one of the following:

[0182] A third replacement module is configured to replace the real identity of the first user with the virtual identity of the first user to carry out the call after the call establishment module 1104 establishes the multimedia call between the first user and the second user, if the real identity of the first user goes offline and the real identity of the first user is in the call before the first user goes offline;

[0183] The fourth replacement module is used to close the real identity call function of the first user after the call establishment module 1104 establishes a multimedia call between the first user and the second user, and determine that the first user is making a call with a virtual identity when the first user's real identity goes offline and the first user's real identity is not in a call before the first user goes offline.

[0184] In one embodiment, the negotiation module 1103 includes at least one of the following:

[0185] A first negotiation unit is configured to perform SDP negotiation based on first real audio media information of the first user, second real audio media information of the second user, a first digital human image of the first user, and a second digital human image of the second user when both the call identities of the first user and the second user are real identities;

[0186] The second negotiation unit is used to perform SDP negotiation based on at least one user's artificial intelligence AI audio and video media information when the call identity of at least one user between the first user and the second user is a virtual identity.

[0187] In one embodiment, the call request carries the first real audio media information;

[0188] The first negotiation unit is specifically configured to:

[0189] According to the first real audio media information, driving the first digital human image corresponding to the first user to move, and obtaining a first digital human video;

[0190] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0191] Sending a first message to the second user, where the first message includes the first audio and video media information and the call identity of the first user;

[0192] receiving a first reply message returned by the second user, where the first reply message includes second real audio media information of the second user;

[0193] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0194] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0195] When the second message is sent to the first user, it is determined that the SDP negotiation is completed. The second message includes the second audio and video media information and the call identity of the second user.

[0196] In one embodiment, the second negotiation unit is specifically configured to:

[0197] When at least one user is the first user, a third message is sent to the second user. The third message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information. The first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0198] receiving a second reply message returned by the second user, where the second reply message includes second real audio media information of the second user;

[0199] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0200] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0201] When the fourth message is sent to the first user, it is determined that the SDP negotiation is completed. The fourth message includes the second audio and video media information and the call identity of the second user.

[0202] In one embodiment, the second negotiation unit is specifically configured to:

[0203] In the case where at least one user is the second user, driving the first digital human image corresponding to the first user to move according to the first real audio media information carried in the call request to obtain the first digital human video;

[0204] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0205] Sending a fifth message to the second user, where the fifth message includes the first audio and video media information and the call identity of the first user;

[0206] Receive a third reply message returned by the second user, the third reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0207] When the sixth message is sent to the first user, it is determined that the SDP negotiation is completed. The sixth message includes the second AI audio and video media information and the call identity of the second user.

[0208] In one embodiment, the second negotiation unit is specifically configured to:

[0209] When at least one user is a first user and a second user, a seventh message is sent to the second user. The seventh message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information. The first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0210] Receive a fourth reply message returned by the second user, the fourth reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0211] When the eighth message is sent to the first user, it is determined that the SDP negotiation is completed. The eighth message includes the second AI audio and video media information and the call identity of the second user.

[0212] In one embodiment, the call establishment module 1104 includes:

[0213] The first receiving unit is configured to receive a first confirmation message sent by the second user when the SDP negotiation is completed;

[0214] A first sending unit, configured to send a first confirmation message to a first user;

[0215] A second receiving unit is configured to receive a second confirmation message sent by the first user after receiving the first confirmation message;

[0216] The establishing unit is configured to complete the establishment of the multimedia call between the first user and the second user upon completing sending the second confirmation message to the second user.

[0217] In one embodiment, the communication device 1100 further includes:

[0218] A second receiving module is configured to receive, after the call establishing module 1104 establishes a multimedia call between the first user and the second user, a first target call request message sent by the first user using the real identity when the first user logs in using the real identity and the first user uses the virtual identity to conduct a multimedia call with the second user, the first target call request message including the third real audio media information of the first user;

[0219] a splitting module, configured to, in response to the first target call request message, split the current call into a sub-call segment from the first user using the virtual identity to the Metaverse server and a sub-call segment from the Metaverse server to the second user;

[0220] A first superposition module is configured to superimpose the third real audio media information onto the digital human video of the first user to generate third audio and video media information;

[0221] A second sending module is used to send a second target call request message to the second user, where the second target call request message includes the third audio and video media information and the call identity of the first user;

[0222] A third receiving module is configured to receive a first response message sent by the second user, where the first response message includes real audio media information or AI audio and video media information of the second user;

[0223] A third sending module is configured to send a target message to the first user, where the target message includes the real audio media information or AI audio and video media information of the second user, and the call identity of the second user;

[0224] The disconnection and establishment module is used to disconnect the sub-call segment of the first user to the Metaverse server with a virtual identity, and to establish a sub-call segment of the first user to the Metaverse server with a real identity.

[0225] The communication device 1100 provided in this embodiment can implement each process of each embodiment of the above-mentioned communication method applied to the metaverse server. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0226] Referring to FIG. 12 , FIG. 12 is a schematic structural diagram of a communication device 1200 provided in an embodiment of the present disclosure, which is applied to a first user terminal where a first user is located. As shown in FIG. 12 , the communication device 1200 includes:

[0227] The first sending module 1201 is used to send a call request to the Metaverse server;

[0228] Among them, the call request is a call request initiated by the first user to the second user. The first user and the second user have both pre-registered real identities and virtual identities. The call request is used to request the Metaverse server to conduct a session description protocol SDP negotiation corresponding to the call identity judgment results based on the call identity judgment results of the first user and the second user, and establish a multimedia call between the first user and the second user after the SDP negotiation is completed.

[0229] In one embodiment, the communication device 1200 further includes:

[0230] The login request sending module is used to send a login request to the Metaverse server with the real identity of the first user before the first sending module 1201 sends a call request to the Metaverse server. The login request is used to request the Metaverse server to replace the first user's virtual identity with the real identity of the first user when the first user's virtual identity is in a call to make a call, or to turn off the first user's virtual identity call function when the first user's virtual identity is not in a call, to determine that the first user is making a call with his real identity.

[0231] In one embodiment, the communication device 1200 further includes:

[0232] The offline request sending module is used to send an offline request to the Metaverse server after the first sending module 1201 sends a call request to the Metaverse server. The offline request is used to request the Metaverse server to replace the real identity of the first user with the virtual identity of the first user to make a call if the real identity of the first user is in a call before receiving the offline request, or to turn off the real identity call function of the first user if the real identity of the first user is not in a call before receiving the offline request, so as to determine that the first user is making a call with a virtual identity.

[0233] The call device 1200 provided in this embodiment is capable of implementing each process of each embodiment of the above-mentioned call method applied to the first user terminal. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.

[0234] The embodiments of the present disclosure also provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, each process of the above-mentioned call method embodiment applied to the metaverse server is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0235] Specifically, referring to FIG. 13 , an embodiment of the present disclosure further provides an electronic device, including a bus 1301 , a transceiver 1302 , an antenna 1303 , a bus interface 1304 , a processor 1305 , and a memory 1306 .

[0236] The processor 1305 is configured to:

[0237] Receiving, via the transceiver 1302, a call request initiated by a first user to a second user, wherein both the first user and the second user have pre-registered real identities and virtual identities;

[0238] In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities;

[0239] Based on the call identity determination results of the first user and the second user, performing a session description protocol SDP negotiation corresponding to the call identity determination results;

[0240] When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

[0241] In one embodiment, the processor 1305, before receiving, through the transceiver 1302, a call request initiated by the first user to the second user, is further configured to perform at least one of the following:

[0242] When a first user logs in with a real identity and the first user's virtual identity is in a call, the first user's real identity replaces the first user's virtual identity to conduct the call, wherein the call request is a request initiated by the first user with the real identity;

[0243] When the first user logs in with a real identity and the first user's virtual identity is not in a call, the first user's virtual identity call function is turned off, and the first user is determined to be making a call with a real identity, wherein the call request is a request initiated by the first user with the real identity.

[0244] In one embodiment, after establishing the multimedia call between the first user and the second user, the processor 1305 is further configured to perform at least one of the following:

[0245] When the real identity of the first user goes offline and the real identity of the first user is in a call before the first user goes offline, the first user's virtual identity replaces the real identity of the first user to conduct the call;

[0246] When the real identity of the first user goes offline and the real identity of the first user is not in a call before the first user goes offline, the real identity call function of the first user is disabled, and the first user is determined to make a call with a virtual identity.

[0247] In one embodiment, the processor 1305 is specifically configured to do at least one of the following:

[0248] When the call identities of the first user and the second user are both real identities, SDP negotiation is performed based on the first real audio media information of the first user, the second real audio media information of the second user, the first digital human image of the first user, and the second digital human image of the second user;

[0249] In a case where the call identity of at least one of the first user and the second user is a virtual identity, SDP negotiation is performed based on at least the artificial intelligence AI audio and video media information of the at least one user.

[0250] In one embodiment, the call request carries the first real audio media information;

[0251] The processor 1305 is specifically configured to:

[0252] According to the first real audio media information, driving the first digital human image corresponding to the first user to move, and obtaining a first digital human video;

[0253] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0254] Sending a first message to the second user, where the first message includes the first audio and video media information and the call identity of the first user;

[0255] receiving a first reply message returned by the second user, where the first reply message includes second real audio media information of the second user;

[0256] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0257] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0258] When the second message is sent to the first user, it is determined that the SDP negotiation is completed. The second message includes the second audio and video media information and the call identity of the second user.

[0259] In one embodiment, the processor 1305 is specifically configured to:

[0260] When at least one user is the first user, a third message is sent to the second user. The third message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information. The first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0261] receiving a second reply message returned by the second user, where the second reply message includes second real audio media information of the second user;

[0262] According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video;

[0263] superimposing the second real audio media information onto the second digital human video to generate second audio and video media information;

[0264] When the fourth message is sent to the first user, it is determined that the SDP negotiation is completed. The fourth message includes the second audio and video media information and the call identity of the second user.

[0265] In one embodiment, the processor 1305 is specifically configured to:

[0266] In the case where at least one user is the second user, driving the first digital human image corresponding to the first user to move according to the first real audio media information carried in the call request to obtain the first digital human video;

[0267] Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information;

[0268] Sending a fifth message to the second user, where the fifth message includes the first audio and video media information and the call identity of the first user;

[0269] Receive a third reply message returned by the second user, the third reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0270] When the sixth message is sent to the first user, it is determined that the SDP negotiation is completed. The sixth message includes the second AI audio and video media information and the call identity of the second user.

[0271] In one embodiment, the processor 1305 is specifically configured to:

[0272] When at least one user is a first user and a second user, a seventh message is sent to the second user. The seventh message includes the first AI audio and video media information of the first user and the call identity of the first user. The call request includes the first AI audio and video media information. The first AI audio and video media information is obtained by superimposing the first user's AI audio onto the first user's digital human video. The first user's digital human video is obtained by driving the first user's digital human image movements using the first user's AI audio.

[0273] Receive a fourth reply message returned by the second user, the fourth reply message including second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the second user's AI audio onto the second user's digital human video, and the second user's digital human video is obtained by driving the second user's digital human image movement via the second user's AI audio;

[0274] When the eighth message is sent to the first user, it is determined that the SDP negotiation is completed. The eighth message includes the second AI audio and video media information and the call identity of the second user.

[0275] In one embodiment, the processor 1305 is specifically configured to:

[0276] When the SDP negotiation is completed, receiving a first confirmation message sent by the second user;

[0277] Sending a first confirmation message to the first user;

[0278] receiving a second confirmation message sent by the first user after receiving the first confirmation message;

[0279] When the second confirmation message is sent to the second user, the multimedia call between the first user and the second user is established.

[0280] In one embodiment, the processor 1305 is further configured to:

[0281] After establishing a multimedia call between a first user and a second user, when the first user logs in with a real identity and conducts a multimedia call with the second user with a virtual identity, receiving a first target call request message sent by the first user with the real identity, the first target call request message including third real audio media information of the first user;

[0282] In response to the first target call request message, the current call is divided into a sub-call segment between the first user using the virtual identity to the Metaverse server and a sub-call segment between the Metaverse server and the second user;

[0283] Superimposing the third real audio media information onto the digital human video of the first user to generate third audio and video media information;

[0284] Sending a second target call request message to the second user, where the second target call request message includes the third audio and video media information and the call identity of the first user;

[0285] Receiving a first response message sent by the second user, where the first response message includes real audio media information or AI audio and video media information of the second user;

[0286] Sending a target message to the first user, where the target message includes the second user's real audio media information or AI audio and video media information, and the second user's call identity;

[0287] Cut off the sub-session segment of the first user using the virtual identity to the Metaverse server, and establish a sub-session segment of the first user using the real identity to the Metaverse server.

[0288] In Figure 13, a bus architecture (represented by bus 1301) is shown. Bus 1301 may include any number of interconnected buses and bridges. Bus 1301 links various circuits together, including one or more processors represented by processor 1305 and memory represented by memory 1306. Bus 1301 may also link various other circuits together, such as peripheral devices, voltage regulators, and power management circuits, all of which are well known in the art and, therefore, will not be described further herein. Bus interface 1304 provides an interface between bus 1301 and transceiver 1302. Transceiver 1302 may be a single component or multiple components, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. Data processed by processor 1305 is transmitted over a wireless medium via antenna 1303. Antenna 1303 also receives data and transmits it to processor 1305.

[0289] The processor 1305 is responsible for managing the bus 1301 and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 1306 may be used to store data used by the processor 1305 when performing operations.

[0290] Optionally, the processor 1305 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0291] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements the various processes of the aforementioned embodiment of the call method applied to the Metaverse server and can achieve the same technical effect. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0292] An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, the various processes of the above-mentioned call method embodiment applied to the first user terminal are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0293] Specifically, as shown in FIG. 14 , an embodiment of the present disclosure further provides an electronic device, including a bus 1401 , a transceiver 1402 , an antenna 1403 , a bus interface 1404 , a processor 1405 , and a memory 1406 .

[0294] The processor 1405 is configured to send a call request to the Metaverse server via the transceiver 1402;

[0295] Among them, the call request is a call request initiated by the first user to the second user. The first user and the second user have both pre-registered real identities and virtual identities. The call request is used to request the Metaverse server to conduct a session description protocol SDP negotiation corresponding to the call identity judgment results based on the call identity judgment results of the first user and the second user, and establish a multimedia call between the first user and the second user after the SDP negotiation is completed.

[0296] In one embodiment, the processor 1405 is further configured to:

[0297] Before sending a call request to the Metaverse server through transceiver 1402, a login request is sent to the Metaverse server using the real identity of the first user through transceiver 1402. The login request is used to request the Metaverse server to replace the first user's virtual identity with the real identity of the first user when the first user's virtual identity is in a call to make a call, or to turn off the first user's virtual identity call function when the first user's virtual identity is not in a call, to ensure that the first user makes a call with his real identity.

[0298] In one embodiment, the processor 1405 is further configured to:

[0299] After sending a call request to the Metaverse server through transceiver 1402, an offline request is sent to the Metaverse server through transceiver 1402. The offline request is used to request the Metaverse server to replace the first user's real identity with the first user's virtual identity to make a call if the first user's real identity is in a call before receiving the offline request, or to turn off the first user's real identity call function if the first user's real identity is not in a call before receiving the offline request, and determine that the first user is making a call with a virtual identity.

[0300] In Figure 14, a bus architecture (represented by bus 1401) is shown. Bus 1401 may include any number of interconnected buses and bridges. Bus 1401 links various circuits together, including one or more processors represented by processor 1405 and memory represented by memory 1406. Bus 1401 may also link various other circuits together, such as peripheral devices, voltage regulators, and power management circuits, all of which are well known in the art and, therefore, will not be described further herein. Bus interface 1404 provides an interface between bus 1401 and transceiver 1402. Transceiver 1402 may be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 1405 is transmitted over a wireless medium via antenna 1403. Antenna 1403 also receives data and transmits it to processor 1405.

[0301] The processor 1405 is responsible for managing the bus 1401 and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 1406 may be used to store data used by the processor 1405 when performing operations.

[0302] Optionally, the processor 1405 may be a CPU, an ASIC, an FPGA, or a CPLD.

[0303] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the aforementioned embodiment of the call method applied to the first user terminal and achieves the same technical effects. To avoid repetition, the description thereof is omitted here. The computer-readable storage medium may be, for example, a ROM, RAM, magnetic disk, or optical disk.

[0304] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0305] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.

[0306] The embodiments of the present disclosure are described above in conjunction with the accompanying drawings, but the present disclosure is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present disclosure, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present disclosure and the claims, all of which are protected by the present disclosure.

Claims

1. A communication method, applied to a Metaverse server, comprising: Receiving a call request initiated by a first user to a second user, wherein both the first user and the second user have pre-registered real identities and virtual identities; In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities; Based on the call identity determination results of the first user and the second user, performing a session description protocol SDP negotiation corresponding to the call identity determination results; When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

2. The method according to claim 1, wherein: Before receiving the call request initiated by the first user to the second user, at least one of the following items is also included: When the first user logs in with a real identity and the first user's virtual identity is in a call, the first user's real identity replaces the first user's virtual identity to conduct the call, wherein the call request is a request initiated by the first user with the real identity; When the first user logs in with the real identity and the first user's virtual identity is not in a call, the virtual identity call function of the first user is turned off, and the first user is determined to make a call with the real identity, wherein the call request is a request initiated by the first user with the real identity.

3. The method according to claim 1 or 2, wherein: After establishing the multimedia call between the first user and the second user, the method further includes at least one of the following: When the real identity of the first user goes offline and the real identity of the first user is in a call before the first user goes offline, replacing the real identity of the first user with the virtual identity of the first user to conduct the call; When the real identity of the first user goes offline and the real identity of the first user is not in a call before the first user goes offline, the real identity call function of the first user is turned off, and the first user is determined to make a call with a virtual identity.

4. The method according to claim 1, wherein: The performing session description protocol SDP negotiation based on the call identity determination result of the first user and the second user includes at least one of the following: When the call identities of the first user and the second user are both real identities, performing SDP negotiation based on the first real audio media information of the first user, the second real audio media information of the second user, the first digital human image of the first user, and the second digital human image of the second user; In the case that the call identity of at least one user between the first user and the second user is a virtual identity, SDP negotiation is performed based at least on the artificial intelligence AI audio and video media information of the at least one user.

5. The method according to claim 4, wherein: The call request carries the first real audio media information; Performing SDP negotiation based on the first real audio media information of the first user, the second real audio media information of the second user, the first digital human image of the first user, and the second digital human image of the second user, including: According to the first real audio media information, driving the first digital human image corresponding to the first user to move, and obtaining a first digital human video; Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information; Sending a first message to the second user, where the first message includes the first audio and video media information and a call identity of the first user; receiving a first reply message returned by the second user, wherein the first reply message includes second real audio media information of the second user; According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video; Superimposing the second real audio media information onto the second digital human video to generate second audio and video media information; When the second message is sent to the first user, it is determined that the SDP negotiation is completed, and the second message includes the second audio and video media information and the call identity of the second user.

6. The method according to claim 4, wherein: The performing SDP negotiation based at least on the artificial intelligence AI audio and video media information of the at least one user includes: In a case where the at least one user is the first user, a third message is sent to the second user, wherein the third message includes first AI audio and video media information of the first user and a call identity of the first user, and the call request includes the first AI audio and video media information, the first AI audio and video media information is obtained by superimposing the AI ​​audio of the first user on the digital human video of the first user, and the digital human video of the first user is obtained by driving the digital human image action of the first user through the AI ​​audio of the first user; receiving a second reply message returned by the second user, where the second reply message includes second real audio media information of the second user; According to the second real audio media information, driving the second digital human image corresponding to the second user to move, and obtaining a second digital human video; Superimposing the second real audio media information onto the second digital human video to generate second audio and video media information; When the fourth message is sent to the first user, it is determined that the SDP negotiation is completed, and the fourth message includes the second audio and video media information and the call identity of the second user.

7. The method according to claim 4, wherein: The performing SDP negotiation based at least on the artificial intelligence AI audio and video media information of the at least one user includes: In the case where the at least one user is the second user, according to the first real audio media information carried in the call request, driving the first digital human image action corresponding to the first user to obtain a first digital human video; Superimposing the first real audio media information onto the first digital human video to generate first audio and video media information; Send a fifth message to the second user, wherein the fifth message includes the first audio and video media information and the The call identity of the first user; receiving a third reply message returned by the second user, wherein the third reply message includes second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the AI ​​audio of the second user onto the digital human video of the second user, and the digital human video of the second user is obtained by driving the digital human image action of the second user by the AI ​​audio of the second user; When the sixth message is sent to the first user, it is determined that the SDP negotiation is completed, and the sixth message includes the second AI audio and video media information and the call identity of the second user.

8. The method according to claim 4, wherein: The performing SDP negotiation based on the digital human image of the at least one user and the artificial intelligence AI audio of the at least one user includes: In a case where the at least one user is the first user and the second user, a seventh message is sent to the second user, wherein the seventh message includes first AI audio and video media information of the first user and a call identity of the first user, and the call request includes the first AI audio and video media information, the first AI audio and video media information is obtained by superimposing the AI ​​audio of the first user on the digital human video of the first user, and the digital human video of the first user is obtained by driving the digital human image action of the first user by the AI ​​audio of the first user; receiving a fourth reply message returned by the second user, wherein the fourth reply message includes second AI audio and video media information of the second user, wherein the second AI audio and video media information is obtained by superimposing the AI ​​audio of the second user onto the digital human video of the second user, and the digital human video of the second user is obtained by driving the digital human image action of the second user by the AI ​​audio of the second user; When the eighth message is sent to the first user, it is determined that the SDP negotiation is completed, and the eighth message includes the second AI audio and video media information and the call identity of the second user.

9. The method according to claim 1, wherein: When the SDP negotiation is completed, establishing a multimedia call between the first user and the second user includes: When the SDP negotiation is completed, receiving a first confirmation message sent by the second user; Sending the first confirmation message to the first user; receiving a second confirmation message sent by the first user after receiving the first confirmation message; When the second confirmation message is sent to the second user, the multimedia call between the first user and the second user is established.

10. The method according to claim 1, wherein: After establishing the multimedia call between the first user and the second user, the method further includes: When the first user logs in with a real identity and the first user conducts a multimedia call with the second user with a virtual identity, receiving a first target call request message sent by the first user with a real identity, wherein the first target call request message includes third real audio media information of the first user; In response to the first target call request message, the current call is divided into a sub-call segment from the first user with the virtual identity to the Metaverse server and a sub-call segment from the Metaverse server to the second user; Superimposing the third real audio media information onto the digital human video of the first user to generate third audio and video media information; Sending a second target call request message to the second user, where the second target call request message includes the third audio and video media information and a call identity identifier of the first user; Receiving a first response message sent by the second user, where the first response message includes real audio media information or AI audio and video media information of the second user; Sending a target message to the first user, where the target message includes real audio media information or AI audio and video media information of the second user, and a call identity identifier of the second user; The sub-session segment of the first user using the virtual identity to the Metaverse server is cut off, and the sub-session segment of the first user using the real identity to the Metaverse server is established.

11. A calling method, applied to a first user terminal where a first user is located, the method comprising: A call request is sent to a Metaverse server, where the call request is a call request initiated by the first user to the second user, wherein the first user and the second user both have pre-registered real identities and virtual identities, and the call request is used to request the Metaverse server to perform a session description protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

12. The method according to claim 11, wherein: Before sending the call request to the Metaverse server, the method further includes: A login request is sent to the Metaverse server with the real identity of the first user, wherein the login request is used to request the Metaverse server to replace the virtual identity of the first user with the real identity of the first user when the virtual identity of the first user is in a call so as to make a call, or to turn off the virtual identity call function of the first user when the virtual identity of the first user is not in a call, so as to determine that the first user is making a call with his real identity.

13. The method according to claim 11 or 12, wherein: After sending the call request to the Metaverse server, the method further includes: A logoff request is sent to the Metaverse server, wherein the logoff request is used to request the Metaverse server to replace the real identity of the first user with the virtual identity of the first user to make a call if the real identity of the first user is in a call before receiving the logoff request, or to disable the real identity call function of the first user if the real identity of the first user is not in a call before receiving the logoff request, and to determine that the first user is making a call with a virtual identity.

14. A communication device, applied to a Metaverse server, comprising: A first receiving module, configured to receive a call request initiated by a first user to a second user, wherein both the first user and the second user are pre-registered with a real identity and a virtual identity; a call identity determination module, configured to determine the call identities of the first user and the second user in response to the call request, wherein the call identities are real identities or virtual identities; A negotiation module, configured to perform a session description protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user; The call establishment module is used to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

15. A communication device, applied to a first user terminal where a first user is located, the device comprising: A first sending module, used to send a call request to a Metaverse server; Among them, the call request is a call request initiated by the first user to the second user. The first user and the second user have both pre-registered real identities and virtual identities. The call request is used to request the Metaverse server to perform a session description protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

16. An electronic device comprising a transceiver and a processor, The processor is configured to: The transceiver receives a call request initiated by a first user to a second user, wherein: The first user and the second user both pre-register with a real identity and a virtual identity; In response to the call request, determining the call identities of the first user and the second user, wherein the call identities are real identities or virtual identities; Based on the call identity determination results of the first user and the second user, performing a session description protocol SDP negotiation corresponding to the call identity determination results; When the SDP negotiation is completed, a multimedia call is established between the first user and the second user.

17. An electronic device comprising a transceiver and a processor, The processor is used to send a call request to the Metaverse server through the transceiver; in, The call request is a call request initiated by a first user to a second user. Both the first user and the second user have pre-registered real identities and virtual identities. The call request is used to request the Metaverse server to perform a Session Description Protocol SDP negotiation corresponding to the call identity judgment result based on the call identity judgment result of the first user and the second user, and to establish a multimedia call between the first user and the second user when the SDP negotiation is completed.

18. An electronic device comprising: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the program implements the steps of the method according to any one of claims 1 to 10, or implements the steps of the method according to any one of claims 11 to 13.

19. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 10, or implements the steps of the method described in any one of claims 11 to 13.

Images