Apparatus, method, and computer program
The method and apparatus address the challenge of establishing IMS sessions for avatar calls by transforming video media into avatar media and using appropriate identifiers, resulting in improved user experience and technical efficiency.
Patent Information
- Application Number
- PCT/EP2024/079144
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-10-27
- Filing Date
- 2024-10-16
- Publication Date
- 2025-05-01
AI Technical Summary
Existing communication systems lack efficient methods for establishing or modifying internet protocol multimedia subsystem (IMS) sessions for avatar calls between user equipment, leading to suboptimal user experience and technical challenges.
A method and apparatus for user equipment to transmit messages to establish or modify IMS sessions for avatar calls, involving the generation and transformation of video media into avatar media, and the use of codec identifiers and avatar identifiers to ensure compatibility and rendering of avatar representations.
The solution enables seamless establishment and modification of IMS sessions for avatar calls, enhancing user experience by ensuring smooth transformation and rendering of avatar media across different user equipment and networks.
Smart Images

Figure EP2024079144_01052025_PF_FP_ABST
Abstract
Description
APPARATUS, METHOD, AND COMPUTER PROGRAMRelated Applications
[0001] This patent application claims the benefit of priority of Indian Patent Application No. 202341073361 filed October 27, 2023, which is hereby incorporated by reference as if reproduced in its entirety.Field
[0002] The present disclosure relates to an apparatus, a method, and a computer program for establishing or modifying an internet protocol multimedia subsystem session between a first user equipment and a second user equipment for an avatar call.Background
[0003] A communication network (i.e. communication system) can be seen as a facility that enables communication sessions between two or more entities such as communication devices, base stations and / or other nodes by providing carriers between the various entities involved in the communications path.
[0004] The communication network may be a wireless communication network. Examples of wireless systems comprise public land mobile networks (PLMN) operating based on radio standards such as those provided by 3GPP, satellite based communication networks and different wireless local networks, for example wireless local area networks (WLAN). The wireless systems can typically be divided into cells, and are therefore often referred to as cellular systems.
[0005] The communication network and associated devices typically operate in accordance with a given standard or specification which sets out what the various entities associated with the system are permitted to do and how that should be achieved. Communication protocols and / or parameters which shall be used for the connection are also typically defined. Examples of standard are the so-called 5G standards.Summary
[0006] According to an aspect there is provided a method for a first user equipment comprising: transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; generating first video media comprising a video of a user; transforming the first video media comprising the video of the user into secondvideo media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to the second user equipment, the second video media or the avatar media.
[0007] Transforming the first video media into the second video media may comprise: decoding the first video media to obtain the video of the user using a video codec; replacing the user in the video of the user with the avatar representation of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user to obtain the second video media using the video codec.
[0008] Transforming the first video media into avatar media may comprise: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user using an avatar codec to obtain the avatar media.
[0009] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise a codec identifier identifying a video codec or an avatar codec; and the method may comprise: receiving, from the second user equipment, a message indicating acceptance of the codec identifier; and transforming the first video media into the second video media or into the avatar media using the video codec or the avatar codec identified by the codec identifier.
[0010] The message to establish or modify an internet protocol multimedia subsystem session for an avatar call between the first user equipment and the second user equipment may comprise an avatar identifier identifying the avatar representation of the user; and the method may comprise: receiving, from the second user equipment, a message indicating an acceptance of the avatar identifier; and transforming the first video media into the second video media or the avatar media using the avatar representation identified by the avatar identifier.
[0011] The avatar identifier may comprise an integer, a string, a universal unique identifier, a uniform resource name or a uniform resource locator.
[0012] The method may comprise: retrieving the avatar representation of the user identified by the avatar identifier from a memory of the first user equipment.
[0013] The method may comprise: receiving, from a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user identified by the avatar identifier.
[0014] The avatar representation of the user identified by the avatar identifier may be received using an internet protocol multimedia subsystem data channel.
[0015] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using an internet protocol multimedia subsystem data channel or using a hypertext transfer protocol.
[0016] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using a session
[0017] The second video media or the avatar media may be transmitted to the second user equipment using a real time protocol.
[0018] According to an aspect there is provided an apparatus for a first user equipment comprising: means for transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; means for generating first video media comprising a video of a user; means for transforming the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and means for transmitting, to the second user equipment, the second video media or the avatar media.
[0019] Transforming the first video media into the second video media may comprise: decoding the first video media to obtain the video of the user using a video codec; replacing the user in the video of the user with the avatar representation of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user to obtain the second video media using the video codec.
[0020] Transforming the first video media into avatar media may comprise: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user using an avatar codec to obtain the avatar media.
[0021] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise a codec identifier identifying a video codec or an avatar codec; and the apparatus may comprise: means for receiving, from the second user equipment, a message indicating acceptance of the codec identifier; and means for transforming the first video media into the second video media or into the avatar media using the video codec or the avatar codec identified by the codec identifier.
[0022] The message to establish or modify an internet protocol multimedia subsystem session for an avatar call between the first user equipment and the second user equipment may comprise an avatar identifier identifying the avatar representation of the user; and the apparatus may comprise: means for receiving, from the second user equipment, a message indicating an acceptance of the avatar identifier; and means for transforming the first video media into the second video media or the avatar media using the avatar representation identified by the avatar identifier.
[0023] The avatar identifier may comprise an integer, a string, a universal unique identifier, a uniform resource name or a uniform resource locator.
[0024] The apparatus may comprise: means for retrieving the avatar representation of the user identified by the avatar identifier from a memory of the first user equipment.
[0025] The apparatus may comprise: means for receiving, from a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user identified by the avatar identifier.
[0026] The avatar representation of the user identified by the avatar identifier may be received using an internet protocol multimedia subsystem data channel.
[0027] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using an internet protocol multimedia subsystem data channel or using a hypertext transfer protocol.
[0028] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using a session description protocol.
[0029] The second video media or the avatar media may be transmitted to the second user equipment using a real time protocol.
[0030] According to an aspect there is provided an apparatus for a first user equipment comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; generating first video media comprising a video of a user; transforming the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user;and transmitting, to the second user equipment, the second video media or the avatar media.
[0031] According to an aspect there is provided an apparatus for a first user equipment comprising circuitry configured to perform: transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; generating first video media comprising a video of a user; transforming the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to the second user equipment, the second video media or the avatar media.
[0032] According to an aspect there is provided a computer program for a first user equipment comprising computer executable code which when run on at least one processor is configured to perform: transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; generating first video media comprising a video of a user; transforming the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to the second user equipment, the second video media or the avatar media.
[0033] According to an aspect there is provided a method for an internet protocol multimedia subsystem comprising: receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to a second user equipment, the second video media or the second avatar media.
[0034] The method may comprise: receiving the avatar representation of the user from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem.
[0035] The avatar representation may be received using an internet protocol multimedia subsystem data channel or a hypertext transfer protocol.
[0036] Transforming the first video media into the second video media may comprise: decoding the first video media to obtain the video of the user using a video codec; replacing the user with the avatar representation of the user in the video of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user using the video codec to obtain the second video media.
[0037] Transforming the first avatar media into the second video media may comprise: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; animating the avatar representation of the user using the gestures of the user within the video of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user using the video codec to obtain the second video media.
[0038] Transforming the first video media into the second avatar media may comprise: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using an avatar codec.
[0039] Transforming the first avatar media into the second avatar media may comprise: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using the avatar codec.
[0040] The method may comprise: receiving, from a network function part of the internet protocol multimedia subsystem network or a network function not part the internet protocol multimedia subsystem network, a message indicating a codec identifier identifying a video codec or an avatar codec; and transforming the first video media or the first avatar media into the second video media or the second avatar media using the video codec or the avatar codec identified by the codec identifier.
[0041] The method may comprise: receiving, from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user; and transforming the first video media or first avatar media into the second video media or the second avatar media using the avatar representation.
[0042] The method may comprise: transmitting, to the first user equipment, the second video media or the second avatar media.
[0043] The second video media or the second avatar media may be transmitted to at least one of the first user equipment or the second user equipment using a real time protocol.
[0044] The gestures of the user within the video of the user and the avatar representation of the user may be transmitted via an internet protocol multimedia subsystem data channel.
[0045] The method may be performed by a media function or a media resource function.
[0046] According to an aspect there is provided an apparatus for a network function inside an internet protocol multimedia subsystem comprising: means for receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; means for transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and means for transmitting, to a second user equipment, the second video media or the second avatar media.
[0047] The apparatus may comprise: means for receiving the avatar representation of the user from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem.
[0048] The avatar representation may be received using an internet protocol multimedia subsystem data channel or a hypertext transfer protocol.
[0049] Transforming the first video media into the second video media may comprise: decoding the first video media to obtain the video of the user using a video codec; replacing the user with the avatar representation of the user in the video of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user using the video codec to obtain the second video media.
[0050] Transforming the first avatar media into the second video media may comprise: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; animating the avatar representation of the user using the gestures of the user within the video of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user using the video codec to obtain the second video media.
[0051] Transforming the first video media into the second avatar media may comprise: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using an avatar codec.
[0052] Transforming the first avatar media into the second avatar media may comprise: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using the avatar codec.
[0053] The apparatus may comprise: means for receiving, from a network function part of the internet protocol multimedia subsystem network or a network function not part the internet protocol multimedia subsystem network, a message indicating a codec identifier identifying a video codec or an avatar codec; and means for transforming the first video media or the first avatar media into the second video media or the second avatar media using the video codec or the avatar codec identified by the codec identifier.
[0054] The apparatus may comprise: means for receiving, from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user; and means for transforming the first video media or first avatar media into the second video media or the second avatar media using the avatar representation.
[0055] The apparatus may comprise: means for transmitting, to the first user equipment, the second video media or the second avatar media.
[0056] The second video media or the second avatar media may be transmitted to at least one of the first user equipment or the second user equipment using a real time protocol.
[0057] The gestures of the user within the video of the user and the avatar representation of the user may be transmitted via an internet protocol multimedia subsystem data channel.
[0058] The apparatus may comprise a media function or a media resource function.
[0059] According to an aspect there is provided an apparatus for a network function inside an internet protocol multimedia subsystem comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to a second user equipment, the second video media or the second avatar media.
[0060] According to an aspect there is provided an apparatus for a network function inside an internet protocol multimedia subsystem comprising circuitry configured to perform:receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to a second user equipment, the second video media or the second avatar media.
[0061] According to an aspect there is provided a computer program for a network function inside an internet protocol multimedia subsystem comprising computer executable code which when run on at least one processor is configured to perform: receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to a second user equipment, the second video media or the second avatar media.
[0062] According to an aspect there is provided a method for a second user equipment comprising: receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; receiving, from the first user equipment, avatar media comprising gestures of the user within a video of the user; decoding the avatar media to obtain the gestures of the user within a video of the user; animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and displaying the video of the avatar representation of the user.
[0063] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise a codec identifier identifying an avatar codec; and the method may comprise: transmitting, to the first user equipment, a message indicating acceptance of the codec identifier; and decoding the avatar media to obtain the gestures of the user within a video of the user using the avatar codec identified by the codec identifying.
[0064] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise an avatar identifier identifying an avatar representation of the user; and themethod may comprise: transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; transmitting, to a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystem network, a message comprising the avatar identifier identifying the avatar representation of the user; and receiving, from the network function part of the internet protocol multimedia subsystem network or the network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user.
[0065] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise an avatar identifier identifying an avatar representation of the user; and the method may comprise: transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; and retrieving the avatar representation of the user identified by the avatar identifier from a memory of the second user equipment.
[0066] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using an internet protocol multimedia subsystem data channel.
[0067] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using a session description protocol.
[0068] The avatar media may be received from the first user equipment using a real time protocol.
[0069] The avatar media comprising gestures of the user within a video of the user may be received via an internet protocol multimedia subsystem data channel.
[0070] According to an aspect there is provided an apparatus for a second user equipment comprising: means for receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; means for receiving, from the first user equipment, avatar media comprising gestures of the user within a video of the user; means for decoding the avatar media to obtain the gestures of the user within a video of the user; means for animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and means for displaying the video of the avatar representation of the user.
[0071] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise a codec identifier identifying an avatar codec; and the apparatus maycomprise: means for transmitting, to the first user equipment, a message indicating acceptance of the codec identifier; and decoding the avatar media to obtain the gestures of the user within a video of the user using the avatar codec identified by the codec identifying.
[0072] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise an avatar identifier identifying an avatar representation of the user; and the apparatus may comprise: means for transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; transmitting, to a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystem network, a message comprising the avatar identifier identifying the avatar representation of the user; and means for receiving, from the network function part of the internet protocol multimedia subsystem network or the network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user.
[0073] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may comprise an avatar identifier identifying an avatar representation of the user; and the method may comprise: transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; and retrieving the avatar representation of the user identified by the avatar identifier from a memory of the second user equipment.
[0074] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using an internet protocol multimedia subsystem data channel.
[0075] The message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call may be transmitted using a session description protocol.
[0076] The avatar media may be received from the first user equipment using a real time protocol.
[0077] The avatar media comprising gestures of the user within a video of the user may be received via an internet protocol multimedia subsystem data channel.
[0078] According to an aspect there is provided an apparatus for a second user equipment comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; receiving, from the first user equipment, avatar mediacomprising gestures of the user within a video of the user; decoding the avatar media to obtain the gestures of the user within a video of the user; animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and displaying the video of the avatar representation of the user.
[0079] According to an aspect there is provided an apparatus for a second user equipment comprising circuitry configured to: receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; receiving, from the first user equipment, avatar media comprising gestures of the user within a video of the user; decoding the avatar media to obtain the gestures of the user within a video of the user; animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and displaying the video of the avatar representation of the user.
[0080] According to an aspect there is provided a computer program for a second user equipment comprising computer executable code which when run on at least one processor is configured to: receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; receiving, from the first user equipment, avatar media comprising gestures of the user within a video of the user; decoding the avatar media to obtain the gestures of the user within a video of the user; animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and displaying the video of the avatar representation of the user.
[0081] According to an aspect, there is provided a computer readable medium comprising program instructions stored thereon for performing at least one of the above methods.
[0082] According to an aspect, there is provided a non-transitory computer readable medium comprising program instructions stored thereon for performing at least one of the above methods.
[0083] According to an aspect, there is provided a non-volatile tangible memory medium comprising program instructions stored thereon for performing at least one of the above methods.
[0084] In the above, many different aspects have been described. It should be appreciated that further aspects may be provided by the combination of any two or more of the aspects described above.
[0085] Various other aspects are also described in the following detailed description and in the attached claims.List of abbreviations
[0086] AF: Application Function
[0087] AMF: Access and Mobility Management Function
[0088] AR: Augmented Reality
[0089] BS: Base Station
[0090] CU: Centralized Unit
[0091] DAC: Digital Asset Container
[0092] DL: Downlink
[0093] DTLS: Datagram Transport Layer Security
[0094] DU: Distributed Unit
[0095] gLTF: graphics Language Transmission Format
[0096] gNB: gNodeB
[0097] HSS: Home Subscriber Server
[0098] IBCF: Interconnection Border Control Function
[0099] I -CSCF: Interrogating Call Session Control Function
[0100] IMS: Internet Protocol Multimedia Subsystem
[0101] IMS AGW: Internet Protocol Multimedia Subsystem Application Level Gateway
[0102] IMS HSS: Internet Protocol Multimedia Subsystem Home Subscriber Server
[0103] LMF: Location Management Function
[0104] LPP: Location Positioning Protocol
[0105] LRDD: Link-based Resource Descriptor Document
[0106] LTE: Long Term Evolution
[0107] MAC: Medium Access Control
[0108] ML: Machine Learning
[0109] MS: Mobile Station
[0110] MTC: Machine Type Communication
[0111] NEF: Network Exposure Function
[0112] NF: Network Function
[0113] NR: New radio
[0114] NRF: Network Repository Function
[0115] NWDAF: Network Data Analytics Function
[0116] P -CSCF: Proxy Call Session Control Function
[0117] PDU: Packet Data Unit
[0118] RAM: Random Access Memory
[0119] (R)AN: (Radio) Access Network
[0120] ROM: Read Only Memory
[0121] RTP: Real Time Protocol
[0122] S -CSCF: Serving Call Session Control Function
[0123] SDP: Session Description Protocol
[0124] SIP: Session Initiation Protocol
[0125] SMF: Session Management Function
[0126] TrGW: Transition Gateway
[0127] TS: Technical Specification
[0128] UE: User Equipment
[0129] URI: Uniform Resource Identifier
[0130] URL: Uniform Resource Locator
[0131] URN: Uniform Resource Name
[0132] UUID: Universal Unique Identifier
[0133] XR: Extended Reality
[0134] 3GPP: 3rdGeneration Partnership Project
[0135] 5G: 5thGeneration
[0136] 5GC: 5G Core network
[0137] 5GS: 5G SystemBrief Description of the Figures
[0138] Embodiments will now be described, by way of example only, with reference to the accompanying Figures in which:
[0139] Fig 1 shows a schematic representation of an originating user equipment communicating with a terminating user equipment via a communication network comprising two internet protocol multimedia subsystem networks in accordance with an example embodiment;
[0140] Fig. 2 shows a schematic representation of an apparatus implementing one or more network functions of an internet protocol multimedia subsystem network;
[0141] Fig. 3 shows a schematic representation of a user equipment;
[0142] Fig. 4 shows a schematic representation of an originating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0143] Fig. 5 shows a schematic representation of another originating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0144] Fig. 6 shows a schematic representation of a terminating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0145] Fig. 7 shows a procedure for a network based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0146] Fig. 8 shows another procedure for a network based rendering for pan avatar call between an originating user equipment and a terminating user equipment;
[0147] Fig. 9 shows a procedure for an originating user equipment based rendering for performing an avatar call between an originating user equipment and a terminating user equipment;
[0148] Fig 10 shows another procedure for an originating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0149] Fig. 11 shows a block diagram of an originating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0150] Fig 12 shows a block diagram of a network based rendering for an avatar call between an originating user equipment and a terminating user equipment;
[0151] Fig 13 shows a block diagram of a terminating user equipment based rendering for an avatar call between an originating user equipment and a terminating user equipment; and
[0152] Fig 14 shows a schematic representation of a non-volatile memory medium storing instructions which when executed by a processor allow a processor to perform one or more of the steps of the methods of Fig. 11 to Fig. 13.Detailed Description of the Figures
[0153] FIG. 1 shows a schematic representation of a first user equipment (denoted UE- A) communicating with a second user equipment comprising a communication network that includes two IMS networks. The communication network may comprise a public land mobile network (PLMN). A communication network may comprise a (radio) access network ((R)AN) (not illustrated), a core network (5GC) (not illustrated) in addition to the two internet protocol multimedia subsystem (IMS) networks shown. The IMS network serving the first user equipment (e.g., IIE-A) may be referred to as an originating IMS network and the IMS network servicing the second user equipment (e.g., IIE-B) may be referred to as a terminating IMS network.
[0154] In this disclosure, the expressions “internet protocol multimedia subsystem” and “internet protocol multimedia subsystem network” may be used interchangeably.
[0155] The (R)AN (not illustrated) may comprise one or more base stations. A base station may be an evolved NodeB (eNB) or a gNodeB (gNB). A gNB may comprise distributed units connected to one or more centralized units. A (base station may be configured to provide wireless connectivity between the first user equipment and a base station and wired and / or wireless connectivity between the base station and the IMS network via the 5GC. and between the second user equipment and the 5GC.
[0156] The 5GC may comprise an access and mobility management function (AMF), a session management function (SMF), an authentication server function (ALISF), a user data management (UDM), a network exposure function (NEF), a unified data repository (UDR), a network repository function (NRF) and / or a network data analytics function (NWDAF), and a user plane comprising a user plane function (UPF). The 5GC may be connected to an IMS network (e.g., the originating IMS network or the terminating IMS network). The 5GC may be to manage traffic and routing from a UE via the (R)AN towards an IMS network (e.g., from the first UE towards the originating IMS network or the second UE towards the terminating IMS network).
[0157] An IMS network (e.g., the originating IMS network or the terminating IMS network) may comprise a proxy call session control function (P-CSCF), a serving call session control function (S-CSCF), an interrogating call session control function (l-CSCF), an IMS application level gateway (IMS-AGW), an IMS home subscriber server (IMS HSS), an IMS application server (IMS AS), a media function (MF) or media repository function (MRF), a transition gateway (TrGW), an interconnection border control function (IBCF), a terminatingIMS core, a data channel signalling function (DCSF), an XR application server (AS), a digital asset container (DAC) or a web server (not represented).
[0158] The DAC may be separate from other network functions of the originating IMS network or may be integrated with other network functions of the originating IMS network. For example, the DAC may be integrated within the IMS HSS or the XR AS.
[0159] Although the DAC is illustrated in Figure 1 as being part of the originating IMS network, it will be understood that the DAC is not necessarily part of the originating IMS network, let alone the communication network (e.g., PLMN). The DAC may be part of the 5GC. The DAC may be separate from other network functions of the 5GC or may be integrated with other networks functions of the 5GC. For example, the DAC may be integrated within the UDM. Alternatively, the DAC may not be part of the communication network (e.g., PLMN). For example, the DAC may be part of a web server or a computing system of a cloud provider that is connected to the communication network (e.g., PLMN).
[0160] Fig. 2 illustrates an example of an apparatus 200 for the IMS illustrated in Fig. 1. The apparatus 200 may comprise at least one random access memory (RAM) 211a, at least on read only memory (ROM) 211 b, at least one processor 212, 213 and an input / output interface 214. The at least one processor 212, 213 may be coupled to the RAM 211a and the ROM 211 b. The at least one processor 212, 213 may be configured to execute an appropriate software code 215. The software code 215 may be for example software code of a network function of an IMS, which when executed by the at least one processor 212, 213 causes the apparatus 200 to perform one or more operations of the network function, including those described in further detail below. The software code 215 may be stored in the ROM 211 b. The apparatus 200 may be interconnected with another apparatus 200 implementing or comprising another network function of the IMS network.
[0161] Fig. 3 illustrates an example of a UE 300, such as the UE-A or the UE-B illustrated on Fig. 1. The UE 300 is a communication device that is capable of (or configured for) sending and receiving radio signals. Non-limiting examples comprise a mobile station (MS) or mobile device such as a mobile phone or what is known as a ’smart phone’, a computer provided with a wireless interface card or other wireless interface facility (e.g., USB dongle), a personal data assistant (PDA) or a tablet provided with wireless communication capabilities, a machine-type communications (MTC) device, a Cellular Internet of things (CloT) device or any combinations of these or the like. The UE 300 may provide, for example, communication of data for carrying communications. The communications maybe one or more of voice, electronic mail (email), text message, multimedia, data, machine data and so on.
[0162] The UE 300 may receive signals over an air or radio interface 307 via appropriate apparatus for receiving and may transmit signals via appropriate apparatus for transmitting radio signals. In Fig. 3 transceiver apparatus is designated schematically by block 306. The transceiver apparatus 306 may be provided for example by means of a radio part and associated antenna arrangement. The antenna arrangement may be arranged internally or externally to the mobile device.
[0163] The UE 300 may be provided with at least one processor 301 , at least one memory ROM 302a, at least one RAM 302b and other possible components 303 for use in software and hardware aided execution of tasks it is designed to perform, including control of access to and communications with access systems and other communication devices. The at least one processor 301 is coupled to the RAM 302b and the ROM 302a. The at least one processor 301 may be configured to execute an appropriate software code 308. The software code 308 may for example allow to perform one or more of the present aspects. The software code 308 may be stored in the ROM 302a.
[0164] The processor, storage and other relevant control apparatus can be provided on an appropriate circuit board and / or in chipsets. This feature is denoted by reference 304. The device may optionally have a user interface such as keypad 305, touch sensitive screen or pad, combinations thereof or the like. Optionally one or more of a display, a speaker and a microphone may be provided depending on the type of the device.
[0165] One or more aspects of this disclosure may establish or modify an IMS session between two communication devices (e.g., a first communication device denoted UE-A and a second communication device denoted UE-B) for an avatar call.
[0166] In this disclosure, the UE-A may be referred to as a first UE or an originating UE and the UE-B may be referred to as a second UE or a terminating UE.
[0167] An avatar call between the UE-A and the UE-B may be a call wherein a user of the UE-A may be represented by an avatar representation in a user interface at the UE-B and another user of the UE-B may be represented by an avatar representation in a user interface at the UE-A.
[0168] In this disclosure the terms “user” and “subscriber” may be used interchangeably.
[0169] In this disclosure the terms “avatar representation” and “avatar” may be used interchangeably.
[0170] An avatar representation of a user may comprise a two dimensional or three dimensional representation of a face, a part of a body or an entire body of the user. The avatar representation of a user may be stored in the DAC.
[0171] An avatar representation of a user may be identified by an avatar identifier. An avatar identifier may comprise a reference or a link to the avatar representation. An avatar identifier may comprise an integer, a string, a universal unique identifier (UIIID) or a uniform resource identifier (URI). A URL may comprise a uniform resource name (URN) or a uniform resource locator (URL). When the avatar identifier is a URI, the avatar identifier may be a dereferenceable URI such that the avatar representation may be easily retrieved.
[0172] Various techniques may be used to retrieve an avatar representation of a user (e.g., by the UE-A, the UE-B or the originating IMS network) as described in further detail below. For example, webfinger, host-meta or link-based resource descriptor discovery (LRDD) may be used to retrieve the avatar representation.
[0173] An avatar representation of a user may be associated with the user or with a user identity of the user. A user may have multiple user identities. For example, a user may have a personal user identity and a professional user identity. That is, a user may be associated with multiple user identities and multiple avatar representations of the user identified by multiple avatar identifiers.An avatar representation of a user may be associated with metadata. The metadata may comprise a format, a resolution, a version, access or usage rights (e.g., times), a date of creation of the avatar representation or licenses to use the avatar representation. A format may comprise graphics language transmission format (gLTF) or virtual reality modelling (VRM).
[0174] An avatar representation of a user may be associated with a human readable description or name. The description or the name may allow the user of the UE-A or the user of the UE-B to identify the avatar representation of the UE-A or the avatar representation of the UE-B.
[0175] One or more aspects of this disclosure enable a first UE (e.g., UE-A) or UE-B to initiate an avatar call with a second UE (e.g., UE-B or UE-A).
[0176] One or more aspects of this disclosure may enable a first UE (e.g., IIE-A), the originating IMS network or the second UE (e.g., UE-B) to retrieve an avatar identifier identifying an avatar representation of a user.
[0177] One or more aspects of this disclosure may enable the first UE (e.g., UE-A), the originating IMS network or the second UE (e.g., UE-B), to provide an avatar representation of a user.
[0178] One or more aspects of this disclosure enable the first UE (e.g., UE-A) and the second UE (e.g., UE-B), together with the originating and terminating IMS networks, to negotiate an avatar representation of a user to be used for an avatar call.
[0179] One or more aspects of this disclosure enable the first UE (e.g., UE-A) and the second UE (e.g., UE-B), together with the originating and terminating IMS networks, to negotiate a codec to be used for an avatar call.
[0180] One or more aspects of this disclosure enable rendering, by the first (e.g., UE-A), the originating IMS network, the terminating IMS network, or the second UE (e.g., UE-B).
[0181] In this disclosure the expression “rendering” may refer to providing a video of an avatar representation of a user or providing gestures of a user within a video of the user and an avatar representation of a user.
[0182] One or more aspects of this disclosure cause the first and second UEs to switch from an avatar call to an audio and / or a video call or vice versa.
[0183] In the following, various implementations are described for negotiating avatar representations and codecs that are used for an avatar call between a first UE (e.g., the UE-A) and the UE-B via a communication network comprising originating and terminating IMS networks.
[0184] Initially, the UE-A may or may not store one or more avatar identifier identifying one or more avatar representation of a user in memory (e.g., local cache). The UE-A may or may not store one or more avatar representations of the user in memory.
[0185] If the UE-A stores one or more avatar identifier identifying one or more avatar representation of the user in memory, the UE-A may select an avatar identifier.
[0186] If the UE-A does not store one or more avatar identifier identifying one or more avatar representation of the user in memory, the UE-A may receive, from the originating IMS network (e.g., IMS AS), an avatar identifier identifying an avatar representation of the user.
[0187] The UE-A may transmit, to the originating IMS network, a message to register UE- A with the originating IMS network.
[0188] The UE-A may transmit, to the UE-B via the originating IMS network (e.g., IMS AS) and the terminating IMS network, a message to establish or modify an IMS session between UE-A and UE-B for an avatar call between the UE-A and the UE-B.
[0189] An IMS session may refer to an IMS data channel established between two communication devices via the originating and terminating IMS networks that may be used for an audio call, a video call and / or avatar call.
[0190] The message may comprise an avatar identifier identifying an avatar representation of a user. The avatar identifier may indicate the intention of the UE-A to establish or modify an IMS session between the UE-A and the UE-B for an avatar call. The avatar identifier may indicate a suggestion to use the avatar representation of the user (by the UE-A and / or the UE-B).
[0191] The message may comprise a codec identifier identifying a video codec or an avatar codec. The codec identifier may indicate a suggestion to use a video codec or an avatar codec for decoding a video media or avatar media before displaying a video of the avatar representation of the user(by the UE-A and / or the UE-B). The video codec or the avatar codec identified by the codec identifier may be based on UE-A capabilities (, a UE- B capabilities or XR application preferences or requirements from the originating and / or terminating IMS network (e.g., originating and / or terminating IMS network prefers or requires a video codec and / or an avatar codec).
[0192] The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter.
[0193] The IMS data channel may comprise one or more IMS application data channels. For example, the IMS data channel may comprise a first IMS application channel between the UE-A and the IMS (e.g., IMS AS or XR AS) and a second IMS application channel between the IIE-B and the IMS (e.g., IMS AS or XR AS).
[0194] In this disclosure, the expressions “IMS application data channel”, “IMS peer-to- application data channel” or “IMS application-to-peer data channel” may be used interchangeably.
[0195] The UE-A may receive, from the UE-B via the originating IMS network (e.g., IMS AS) and the terminating IMS network, a message indicating acceptance of the avatar identifier and the codec identifier. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters.
[0196] £ - originating UE based rendering implementation
[0197] In an originating UE based rendering implementation, an originating UE (e.g., UE- A) may perform rendering.
[0198] If the originating UE (e.g., UE-A) stores the avatar representation of the user of the originating UE (e.g., UE-A) (selected by the user of the originating UE (e.g., UE-A) as described in detailed above) in memory, the UE-A may retrieve the avatar representation of the user.
[0199] If the originating UE (e.g., UE-A) does not store the avatar representation of the user in memory, the originating UE (e.g., UE-A) may retrieve, from a network function of the originating IMS network (e.g., XR AS of the originating IMS network), the avatar representation of the user.
[0200] The originating UE (e.g., UE-A) may capture a video of a user or receive a video of the user from a camera connected to the originating UE (e.g., UE-A). The originating UE (e.g., UE-A) may encode the video of the user using the video codec to generate video media comprising the video of the user.
[0201] I. a - video media to video media transcoding
[0202] If the codec identifier identifies a video codec, the originating UE (e.g., IIE-A) may transform (e.g., transcode) the video media comprising the video of the user into video media comprising a video of an avatar representation of the user using the video codec.
[0203] The originating UE (e.g., UE-A) may decode the video media using the video codec to obtain the video of the user. The originating UE (e.g., UE-A) may replace the user with the avatar representation of the user to obtain a video of the avatar representation of the user.
[0204] The originating UE (e.g., UE-A) may display the video of the avatar representation of the user (e.g., thumbnail view).
[0205] The originating UE (e.g., UE-A) may encode the video of the avatar representation of the user using the video codec to obtain the video media comprising the video of the avatar representation of the user.
[0206] The originating UE (e.g., UE-A may transmit, to the UE-B via the originating and the terminating IMS networks, the video media comprising the video of the avatar representation of the user. The video media may be transmitted using a real time protocol (RTP) or a datagram transport layer security (DTLS).
[0207] The terminating UE (e.g., UE-B) may decode the video media comprising the video of the avatar representation of the user of originating UE (e.g., UE-A) using the video codec. The UE-B may display the video of the avatar representation of the user of the originating UE (e.g., UE-A.
[0208] l.b - video media to avatar media transcoding
[0209] If the codec identifier identifies an avatar video codec, the originating UE (e.g., UE- A) may transform (i.e. transcode) the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation using the avatar codec.The originating UE (e.g., UE-A) may decode the video media using the video codec to obtain the video of the user. The originating UE (e.g., UE-A) may determine gestures of the user within the video of the user. For example, the originating UE (e.g., UE-A) may use artificial intelligence or machine learning to perform detection or recognition techniques and determine gestures (e.g. movements, poses or expressions) of the user within the video of the user.
[0210] The originating UE (e.g., IIE-A) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The originating UE (e.g., UE-A) may display the video of the avatar representation of the user (e.g., thumbnail view).
[0211] The originating UE (e.g., UE-A) may encode the gestures of the user within the video of the user and the avatar representation using the avatar codec to obtain avatar media comprising the gestures of the user within the video of the user.
[0212] The originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP or DTLS.
[0213] The terminating UE (e.g., UE-B may decode the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user using the avatar codec. The UE-B may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The UE-B may display video of the avatar representation of the user.
[0214] II - IMS network based rendering implementation
[0215] In an IMS network based rendering implementation, the originating and / or terminating IMS network (e.g., MF or MRF or another network function of the originating and / or terminating IMS network) may perform rendering.
[0216] If the MF or MRF of the originating IMS network stores the avatar representation of the user, in memory the MF or MRF of the originating IMS network may retrieve the avatar representation of the user from memory.
[0217] If the MF or MRF of the originating IMS network does not store the avatar representation of the user in memory, the MF or MRF of the originating IMS network may receive, from another network function of the originating IMS network (e.g., XR AS), the avatar representation of the user.
[0218] The originating UE (e.g., UE-A) may capture a video of a user or receive video of a user captured by at least one camera that is connected to the originating UE (e.g., UE-A). The UE-A may generate video media comprising the video of the user using a video codec.The originating UE (e.g., IIE-A) may transmit, to the originating IMS network, the video media comprising the video of the user. The video media may be transmitted using RTP or DTLS.
[0219] Alternatively, the originating UE (e.g., UE-A) may determine gestures of the user within the video of the user. For example, the UE-A may process the video to determine the gestures performed by the user in the video. The originating UE (e.g., UE-A) may process the video using an artificial intelligence or machine learning model configured to perform gesture detection or gesture recognition to identify gestures of the user within the video of the user. The originating UE (e.g., UE-A) may generate avatar media comprising the gestures of the user within the video of the user using an avatar codec. The originating UE (e.g., UE-A) may transmit, to the originating IMS network, the avatar media comprising the gestures of the user within the video of the user. The avatar media may be transmitted using RTP.
[0220] The MF or MRF of the originating IMS network may receive the video media comprising the video of the user. Alternatively, the MF or MRF of the originating IMS network may receive the avatar media comprising the gestures of the user within the video of the user.
[0221] 11. a - video media or avatar media to video media transcoding
[0222] If the codec identifier identifies a video codec, the MF or MRF of the originating IMS network may transform (e.g., transcode) the video media comprising the video of the user into video media comprising a video of an avatar representation of the user using the video codec.
[0223] The MF or MRF of the originating IMS network may decode the video media using a video codec to obtain the video of the user using the video codec. The MF or MRF of the originating IMS network may replace the user with the avatar representation of the user to obtain a video of the avatar representation of the user. The MF or MRF of the originating IMS network may encode the video of the avatar representation of the user using the video codec to obtain the video media comprising the video of the avatar representation of the user.
[0224] The MF or MRF of the originating IMS network may transmit, to the UE-B via the terminating IMS network, the video media comprising the video of the avatar representationof the user of the originating UE (e.g., IIE-A). The video media may be transmitted using RTP or DTLS.
[0225] The terminating UE (e.g., UE-B) may decode the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A) using the video codec to obtain the video of the avatar representation of the user of the originating UE (e.g., UE-A). The terminating UE (e.g., UE-B) may display the video of the avatar representation of the user of the originating UE (e.g., UE-A).
[0226] The MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A). The video media may be transmitted using RTP or DTLS.
[0227] The UE-A may decode the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A) using the video codec to obtain the video of the avatar representation of the user of the originating UE (e.g., UE-A). The originating UE (e.g., UE-A) may display the video of the avatar representation of the user (e.g., thumbnail view) of the originating UE (e.g., UE-A).
[0228] Alternatively, the MF or MRF of the originating IMS network may transform (e.g., transcode) the avatar media comprising the gestures of the user within the video of the user of the originating UE (e.g., UE-A) into video media comprising a video of an avatar representation of the user of the originating UE (e.g., UE-A) using the video codec.
[0229] The MF or MRF of the originating IMS network may decode the avatar media using an avatar codec to obtain the gestures of the user of the originating UE (e.g., UE-A) within the video of the user using the video codec. The MF or MRF of the originating IMS network may animate the avatar representation of the user using the gestures of the user within the video of the user of the originating UE (e.g., UE-A) to obtain a video of an avatar representation of the user of the originating UE (e.g., UE-A). The MF or MRF of the originating IMS network may encode the video of the avatar representation of the user of the originating UE (e.g., UE-A) using the video codec to obtain the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A).
[0230] The MF or MRF of the originating IMS network may transmit to the terminating UE (e.g., UE-B) via the terminating IMS network, the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A).
[0231] The terminating UE (e.g., IIE-B) may decode the video media comprising the video comprising the avatar representation of the user of the originating UE (e.g., UE-A) using the video codec to obtain the video comprising the avatar representation of the user of the originating UE (e.g., UE-A). The terminating UE (e.g., UE-B) may display the video comprising the avatar representation of the user of UE-A.
[0232] The MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the video media comprising the video of the avatar representation of the user of the originating UE (e.g., UE-A). The video media may be transmitted using RTP or DTLS.
[0233] The originating UE (e.g., UE-A) may decode the video media comprising the video of the avatar representation of the user using the video codec to obtain the video of the avatar representation of the user. The originating UE (e.g., UE-A) may display the video of the avatar representation of the user (e.g., thumbnail view).
[0234] ll-b - video media or avatar media to avatar media transcoding
[0235] If the codec identifier identifies an avatar video codec, the MF or MRF of the originating IMS network may transform (|e.g., transcode) the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and an avatar representation using the avatar codec.
[0236] The MF or MRF of the originating IMS network may decode the video media using a video codec to obtain the video of the user. The MF or MRF of the originating IMS network may determine gestures of the user within the video of the user. For example, the MF or MRF of the originating IMS network may use artificial intelligence or machine learning to perform detection or recognition techniques and determine gestures of the user within the video of the user. The MF or MRF of the originating IMS network may encode the gestures of the user within the video of the user and the avatar representation using the avatar codec to obtain avatar media comprising the gestures of the user within the video of the user.
[0237] The MF or MRF of the originating IMS network may transmit to the terminating UE (e.g., UE-B) via the terminating IMS network, the avatar media comprising the gestures of the user of the originating UE (e.g., UE-A) within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP or DTLS.
[0238] The terminating UE (e.g., UE-B) may decode the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the userusing the avatar codec to obtain the gestures of the user within the video of the user and the avatar representation of the user. The terminating UE (e.g., IIE-B) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The terminating UE (e.g., UE-B) may display the video of the avatar representation of the user.
[0239] The MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the avatar media comprising the gestures of the user of the originating UE (e.g., UE-A) within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP or DTLS.
[0240] The originating UE (e.g., UE-A) may decode the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user using the avatar codec to obtain the gestures of the user within the video of the user and the avatar representation of the user. The originating UE (e.g., UE-A) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The originating UE (e.g., UE-A) may display video of the avatar representation of the user (e.g., thumbnail view).
[0241] Alternatively, the MF or MRF of the originating IMS network may transform (i.e. transcode) the avatar media comprising the gestures of the user within the video of the user into avatar media comprising the gestures of the user within the video of the user and an avatar representation using the avatar codec.
[0242] The MF or MRF of the originating IMS network may decode the avatar media using the avatar codec to obtain the gestures of the user within the video of the user. The MF or MRF of the originating IMS network may encode the gestures of the user within the video of the user and the avatar representation of the suer using the avatar codec to obtain avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user.
[0243] The MF or MRF of the originating IMS network may transmit to the terminating UE (e.g., UE-B) via the terminating IMS network, the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP or DTLS.
[0244] The terminating UE (e.g., UE-B) may decode the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the userusing the avatar codec. The terminating UE (e.g., IIE-B) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The terminating UE (e.g., UE-B) may display the video of the avatar representation of the user.
[0245] The MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be encrypted using RTP or DTLS.
[0246] The originating UE (e.g., UE-A) may decode the avatar media comprising the gestures of the user within the video of the user and the avatar representation of the user using the avatar codec. The originating UE (e.g., UE-A) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The originating UE (e.g., UE-A) may display video of the avatar representation of the user (e.g., thumbnail view).
[0247] III - Terminating UE based rendering implementation
[0248] In a terminating UE based rendering implementation, a terminating UE (e.g., UE- B) may perform the rendering.
[0249] If the terminating UE (e.g., UE-B) stores the avatar representation of the user in memory, the terminating UE (e.g., UE-B) may retrieve the avatar representation of the user from memory.
[0250] If the terminating UE (e.g., UE-B) does not store the avatar representation of the user in memory, the terminating UE (e.g., UE-B) may receive, from the originating IMS network (e.g., XR AS) via the terminating IMS network, the avatar representation of the user.
[0251] The originating UE (e.g., UE-A) may capture a video of a user or receive a video of the user from a camera connected to the originating UE (e.g., UE-A).
[0252] If the codec identifier identifies an avatar codec, the originating UE (e.g., UE-A) may determine gestures of the user within the video of the user. For example, the originating UE (e.g., UE-A_ may use artificial intelligence or machine learning to perform detection or recognition techniques and determine gestures of the user within the video of the user.
[0253] The originating UE (e.g., IIE-A) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user. The originating UE (e.g., UE-A) may display the video of the avatar representation of the user.
[0254] The originating UE (e.g., UE-A) may encode the gestures of the user within the video of the user using the avatar codec to obtain avatar media comprising the gestures of the user within the video of the user.
[0255] The originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, the avatar media comprising the gestures of the user within the video of the user. The avatar media may be transmitted using RTP or DTLS.
[0256] The terminating UE (e.g., UE-B may) decode the avatar media comprising the gestures of the user within the video of the user using the avatar codec to obtain the gestures of the user within the video of the user. The terminating UE (e.g., UE-B) may mix (combine) the gestures of the user within the video of the user the avatar representation of the user.
[0257] The terminating UE (e.g., UE-B) may animate the avatar representation of the user using the gestures of the user within the video of the user to obtain the video of the avatar representation of the user. The terminating UE (e.g., UE-B) may display the video of the avatar representation of the user. The avatar media may be transmitted using RTP or DTLS.
[0258] It will be understood that, in the originating UE based rendering implementation, the IMS network based rendering implementation, or the terminating UE based rendering implementation, the roles of the originating UE (e.g., UE-A) and the terminating UE (e.g., UE-B) may be inverted.
[0259] It will be understood that, In the originating UE based rendering implementation, the IMS network based rendering implementation or the terminating UE based rendering implementation, the originating UE (e.g., UE-A), the originating IMS network, the terminating IMS network or the terminating UE (e.g., UE-B) may modify the IMS session to switch from the avatar call to the audio and video call at any time, for example using a re-invite or update message. The originating UE (e.g., UE-A), the originating IMS network, the terminating IMS network or the terminating UE (e.g., UE-B) may switch from the avatar call to the audio / videocall at any time based on a request from a user, from the originating IMS network (e.g., XR AS) or from the terminating IMS network.
[0260] Fig 4 shows a schematic representation of another example of a procedure for the “originating UE" based implementation (e.g., video media to video media transcoding).
[0261] At step 1, the originating UE (e.g., UE-A) may receive, from the originating IMS network (e.g., XR AS), the avatar representation of the user.
[0262] At step 2, the originating UE (e.g., UE-A) may perform rendering. The originating UE (e.g., UE-A) may transform (e.g., transcode) the video media comprising the video of the user into video media comprising the video of the avatar representation of the user.
[0263] At step 3, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating IMS network and the terminating IMS network, the video media comprising the video of the avatar representation of the user.
[0264] Fig 5 shows a schematic representation of an example of the “originating UE" based implementation (i.e. I.b - video media to avatar media transcoding).
[0265] At step 1, the originating UE (e.g., UE-A) may receive, from the originating IMS network (e.g., XR AS), the avatar representation of the user.
[0266] At step 2, the originating UE (e.g., UE-A) may perform rendering. The originating UE (e.g., UE-A) may transform (e.g., transcode) the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user.
[0267] At step 3, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating IMS network and the terminating ISM network, the avatar media comprising gestures of the user with the video of the user and the avatar representation of the user.
[0268] Fig. 6 shows a schematic representation of an example procedure of the “terminating UE" based implementation (e.g., avatar media to avatar media transcoding).
[0269] At step 1 , the terminating UE (e.g., UE-B) may receive, from the originating IMS network (e.g., XR AS) via the terminating IMS network, the avatar representation of the user.
[0270] At step 2, the originating UE (e.g., IIE-A) may transform (e.g., transcode) video media comprising video of the user into avatar media comprising gestures of the user within the video of the user.
[0271] At step 3, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating IMS network and the terminating IMS network, the avatar media comprising gestures of the user with the video of the user.
[0272] At step 4, the terminating UE (e.g., UE-B) may decode the avatar media comprising gestures of the user with the video of the user. The terminating UE (e.g., UE-B) may combine the gestures of the user with the video of the user and the avatar representation of the user.
[0273] Fig. 7 shows a procedure for the network based rendering implementation.
[0274] At step 1 , an IMS session may be established between the originating UE (e.g., UE-A) and the terminating UE (e.g., UE-B). A bootstrap data channel may be established for both the originating UE (e.g., UE-A) and terminating UE (e.g., UE-B).
[0275] At step 2, the originating UE (e.g., UE-A) may decide to request network based rendering (i.e. network based rendering implementation) based on a status of the originating UE (e.g., UE-A) (e.g., power, signal, computing power, internal storage, etc).
[0276] At step 3, the originating UE (e.g., UE-A) and the originating IMS network (e.g., XR AS) may perform rendering negotiation.
[0277] At step 4, if the rendering negotiation result is successful at step 3, the originating UE (e.g., UE-A) may initiate peer to application data channels, which may be used for XR data transmission between the originating UE (e.g., UE-A) and the originating IMS network. During the peer to application data channel establishment procedure, the DCSF of the originating IMS network may instruct the MF or MRF of the originating IMS network via the IMS AS of the originating IMS network how to establish the peer to application data channel and corresponding media processing specification.
[0278] At step 5, the originating UE (e.g., UE-A) and the IMS AS of the originating IMS network may perform a media re-negotiation, to connect the originating UE’s, audio and video media stream to MF or MRF of the originating IMS network. The originating UE (e.g., UE-A) may transmit a message comprising an avatar identifier identifying an avatar representation of the user and a codec identifier identifying a video codec or an avatarcodec. The media re-negotiation request may be transmitted using a peer to application data channel.
[0279] At step 6, the terminating UE (e.g., IIE-B) and the IMS AS of the originating IMS network may perform a media re-negotiation to connect the terminating UE’s (e.g., LIE-B's) audio and video media stream to the MF or MRF of the originating IMS network.
[0280] The IMS of the originating IMS network (e.g., IMS AS), may transmit to the terminating UE (e.g., UE-B) via the terminating IMS network, a message comprising the avatar identifier and the codec identifier using a peer to application data channel. The terminating UE (e.g., UE-B) may accept or reject the audio or video session. If the terminating UE (e.g., UE-B) rejects the audio or video session, the terminating UE (e.g., UE-B) may terminate the audio or video session. The terminating UE (e.g., UE-B) may accept or reject the avatar identifier or the codec identifier. If the terminating UE (e.g., UE- B) rejects the avatar identifier or the codec identifier, the terminating UE (e.g., UE-B) may terminate the audio or video session.
[0281] At step 7, the XR AS of the originating IMS network may transmit, to the DAC, a message comprising the avatar identifier. The message may comprise a UE identifier identifying the originating UE (e.g., UE-A).
[0282] At step 8, the XR AS of the originating IMS network may receive, from the DAC, a message comprising the avatar representation of the user. The avatar representation of the user may be identified based on the avatar identifier. The avatar representation of the user may be identified based on the user identifier.
[0283] At step 9, the XR AS of the originating IMS network may start Rendering.
[0284] At step 10, the XR AS of the originating IMS network may transmit to the MF or MRF of the originating IMS network, a message comprising the avatar representation of the user.
[0285] At step 11 , the originating UE (e.g., UE-A) may transmit to the MF or MRF of the originating IMS network, video media comprising a video of a user or avatar medio comprising gestures of the user within the video of the user.
[0286] At step 12, the MF or MRF of the originating IMS network may perform rendering.
[0287] The MF or MR F of the originating IMS network may transform the video media comprising the video of the user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0288] Alternatively, the MF or MRF of the originating IMS network may transform the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0289] Alternatively, the MF or MRF of the originating IMS network may transform the avatar media comprising gestures of the user within the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0290] Alternatively, the MF or MRF of the originating IMS network may transform the avatar media comprising gestures of the user within the video of the user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0291] At step 13, the MF or MRF of the originating IMS network may transmit to the terminating UE (e.g., IIE-B) via the terminating IMS network, the video media comprising the video of the avatar representation of the user. The video media may be transmitted using RTP.
[0292] Alternatively, the MF or MRF of the originating IMS network may transmit to the terminating UE (e.g., UE-B), the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP.
[0293] At step 14, the MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the video media comprising the video of the avatar representation of the user. The video media may be transmitted using RTP.
[0294] Alternatively, the MF or MRF of the originating IMS network may transmit to the originating UE (e.g., UE-A), the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP.
[0295] Fig 8 shows a procedure for the network based rendering implementation.
[0296] At step 1 , the originating UE (e.g., IIE-A) may decide to request network based rendering based on a status or capabilities of the originating UE (e.g., UE-A) (e.g., power, signal, computing power, internal storage, etc).
[0297] At step 2, the originating UE (e.g., UE-A) may transmit to the IMS AS of the originating IMS network via the P-CSCF, the l-CSCF, the S-CSF or the IMS AGW of the originating IMS network, a message comprising an avatar identifier identifying an avatar representation of a user and a codec identifier a video codec or an avatar codec. The message may comprise metadata associated with the avatar representation. The message may comprise a description or name associated with avatar representation. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter. The codec identifier or the avatar identifier may be conveyed via a m- or a- line.
[0298] At step 3, the IMS AS of the originating IMS network may transmit to the XR AS of the originating IMS network, a message comprising an avatar identifier identifying an avatar representation of a user and a codec identifier a video codec or an avatar codec. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter. The message may cause the XR AS of the originating IMS network to retrieve the avatar representation from the DAC.
[0299] At step 4, the IMS AS of the originating IMS network may transmit to the terminating UE (e.g. UE-B), a message comprising an avatar identifier identifying an avatar representation of a user and a codec identifier a video codec or an avatar codec. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter. The terminating UE (e.g., UE- B) may accept or reject the avatar identifier or the codec identifier. If the terminating UE(e.g., UE-B) rejects the avatar identifier or the codec identifier, the terminating UE (e.g., UE- B) may terminate the session.
[0300] At step 5, the XR AS of the originating IMS network may transmit, to the DAC, a message to receive an avatar representation of a user. The message may comprise an avatar identifier identifying the avatar representation of the user. The message may comprise a user identifier of the user.
[0301] At step 6, the DAC may transmit to the XR AS of the originating IMS network, a message comprising the avatar representation of the user. The avatar representation of the user may be identified based on the avatar identifier. The avatar representation of the user may be identified based on the user identifier.
[0302] At step 7, the terminating UE (e.g., UE-B) may transmit to the IMS AS of the originating IMS network, a message comprising an indication of acceptance of the avatar identifier and the codec identifier.
[0303] At step 8, the IMS AS of the originating IMS network may transmit to the originating UE (e.g. UE-A), a message comprising an indication of acceptance of the avatar identifier and the codec identifier.
[0304] At step 9, the IMS AS of the originating IMS network may transmit to the XR AS of the originating IMS network, a message comprising the avatar identifier and the codec identifier. The message may cause the XR AS of the originating IMS network to start rendering.
[0305] At step 10, the XR AS of the originating IMS network may start rendering.
[0306] At step 11 , the XR AS of the originating IMS network may transmit to the MF or MRF of the originating IMS network, a message comprising the avatar representation of the user.
[0307] At step 12, the originating UE (e.g., UE-A) may transmit to the MF or MRF of the originating IMS network, video media comprising a video of a user or avatar media comprising gestures of the user within the video of the user.
[0308] At step 13, the MF or MRF of the originating IMS network may perform rendering.
[0309] The MF or MR F of the originating IMS network may transform the video media comprising the video of the user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0310] Alternatively, the MF or MRF of the originating IMS network may transform the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0311] Alternatively, the MF or MRF of the originating IMS network may transform the avatar media comprising gestures of the user within the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0312] Alternatively, the MF or MRF m of the originating IMS network ay transform the avatar media comprising gestures of the user within the video of the user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0313] At step 14, the MF or MRF of the originating IMS network may transmit, to the terminating UE (e.g. IIE-B) via the terminating IMS network, the video media comprising the video of the avatar representation of the user. The video media may be transmitted using RTP.
[0314] Alternatively, the MF or MRF of the originating IMS network may transmit, to the terminating UE (e.g. UE-B), the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP.
[0315] At step 15, the MF or MRF of the originating IMS network may transmit, to the originating UE (e.g., UE-A), the video media comprising the video of the avatar representation of the user. The video media may be transmitted using RTP.
[0316] Alternatively, the MF or MRF of the originating IMS network may transmit, to the originating UE (e.g., UE-A) via the IMS, the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The avatar media may be transmitted using RTP.
[0317] Fig. 9 shows a procedure for the originating UE based rendering implementation.
[0318] At step 1 , an IMS session may be established between the originating UE (e.g., UE-A) and the terminating UE (e.g., UE-B). A bootstrap data channel may be established for both the originating UE (e.g., UE-A) and the terminating UE (e.g., UE-B).
[0319] At step 2, the originating UE (e.g., UE-A) may upgrade the IMS session with XR experience.
[0320] The originating UE (e.g., UE-A) may initiate a re-INVITE adding media descriptors required by the XR application to be established end to end (e.g., RTP and / or application data channels). Application data channels may be anchored in the MF or the MRF of the originating IMS network.
[0321] The originating UE (e.g., UE-A) may transmit to the XR AS of the originating IMS network, a message comprising an avatar identifier identifying a representation of the user. The message may comprise a codec identifier identifying a video codec or an avatar codec. The message may comprise a user identifier identifying the user. The message may be transmitted using an application data channel.
[0322] At step 3, the XR AS of the originating IMS network may transmit, to the DAC, a message to request an avatar representation of the user. The message may comprise the avatar identifier identifying the avatar representation of the user. The message may comprise the user identifier identifying the user.
[0323] At step 4, the DAC may transmit, to the XR application server of the originating IMS network, a message comprising the avatar representation of the user.
[0324] At step 5, the XR AS of the originating IMS network may transmit to the originating UE (e.g., UE-A), a message comprising the avatar representation of the user. The message may comprise the avatar identifier identifying the avatar representation of the user. The message may comprise the user identifier identifying the user. The message may be transmitted using an application data channel, using a hypertext transfer protocol (HTTP), using a websocket or using a SIP.
[0325] At step 6, the originating UE (e.g., UE-A) may transmit to the XR AS of the originating IMS network, a message indicating success or failure.
[0326] At step 7, the originating UE (e.g., UE-A) may transform video media comprising a video of a user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0327] Alternatively, the originating UE (e.g., IIE-A) may transform the video media comprising the video of the user into avatar media comprising gestures of the user within the video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0328] At step 8, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, the video media comprising the video of the avatar representation of the user or the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The video media may be received using RTP.
[0329] At step 9, the originating UE (e.g., UE-A) may receive, from the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, video media comprising a video of an avatar representation of another user. The video media may be received using RTP.
[0330] Alternatively, the originating UE (e.g., UE-A) may receive, from the terminating UE (e.g., UE-B), avatar media comprising gestures of another user within a video of the other user and the avatar representation of the other user. The avatar media may be received using RTP.
[0331] At step 10, the originating UE (e.g., UE-A) may decode the video media comprising a video of the avatar representation of the other user using a video codec. The originating UE (e.g., UE-A) may display the video of the avatar representation of the other user.
[0332] Alternatively, the originating UE (e.g., UE-A) may decode the avatar media comprising avatar media comprising gestures of the other user within the video of the other user and the avatar representation of the other user. The originating UE (e.g., UE-A) may animate the avatar representation of the other user using the gestures of the other user to obtain a video of the avatar representation of the other user. The originating UE (e.g., UE- A) may display the video of the avatar representation of the other user.
[0333] Fig. 10 shows a procedure for the originating UE based rendering implementation.
[0334] At step 1 , the originating UE (e.g., UE-A) may decide to request originating UE based rendering based on a status or capabilities of the originating UE (e.g., UE-A) (e.g., power, signal, computing power, internal storage, etc).
[0335] At step 2, the originating UE (e.g., IIE-A) may transmit to the originating IMS network (e.g., IMS AS via the P-CSCF, the l-CSCF, the S-CSF or the IMS AGW of the originating IMS network), a message comprising an avatar identifier identifying an avatar representation of a user and a codec identifier a video codec or an avatar codec. The message may comprise metadata associated with the avatar representation. The message may comprise a description or name associated with avatar representation. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter.
[0336] At step 3, the IMS AS of the originating IMS network may transmit to the XR AS of the originating IMS network, a message comprising the avatar identifier identifying the avatar representation of the user and the codec identifier a video codec or an avatar codec. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SIP parameter or a SDP parameter. The message may cause the XR AS of the originating IMS network to retrieve the avatar representation from the DAC.
[0337] At step 4, the originating IMS network (e.g., the IMS AS) may transmit to the terminating UE (e.g., UE-B), a message comprising the avatar identifier identifying the avatar representation of the user and the codec identifier identifying a video codec or an avatar codec. The message may be transmitted using an IMS data channel, using SIP or using SDP. The message may comprise an SIP message or the message may comprise SDP content (e.g., SDP offer). The SIP message may comprise one or more SIP parameters or SDP content may comprise one or more SDP parameters. The codec identifier or the avatar identifier may be transmitted as a SI P parameter or a SDP parameter. The terminating UE (e.g., UE-B) may accept or reject the avatar identifier or the codec identifier. If the terminating UE (e.g., UE-B) rejects the avatar identifier or the codec identifier, the terminating UE (e.g., UE-B) may terminate the session.
[0338] It will be understood the terminating UE (e.g., UE-B) may reject the avatar call by rejecting the avatar identifier or the codec identifier. Alternatively, the terminating UE (e.g., UE-B) may reject the avatar call by transmitting to the IMS AS of the originating IMSnetwork, an SIP message (e.g., re-invite) without an avatar identifier identifying an avatar representation of a user.
[0339] At step 5, the XR AS of the originating IMS network may transmit, to the DAC, a message to receive an avatar representation of a user. The message may comprise an avatar identifier identifying the avatar representation of the user. The message may comprise a user identifier of the user.
[0340] At step 6, the DAC may transmit to the XR AS of the originating IMS network, a message comprising the avatar representation of the user. The avatar representation of the user may be identified based on the avatar identifier. The avatar representation of the user may be identified based on the user identifier.
[0341] At step 7, the terminating UE (e.g., IIE-B) may transmit to the IMS AS of the originating IMS network, a message comprising an indication of acceptance of the avatar identifier and the codec identifier.
[0342] At step 8, the IMS AS of the originating IMS network may transmit to the originating UE (e.g., UE-A), a message comprising an indication of acceptance of the avatar identifier and the codec identifier.
[0343] At step 9, the originating UE (e.g., UE-A) may transmit to the XR AS of the originating IMS network, a message comprising the avatar identifier and the codec identifier. The message may cause the XR AS of the originating IMS network to start rendering.
[0344] At step 10, the XR AS of the originating IMS network may start rendering.
[0345] At step 11 , the XR AS of the originating IMS network may transmit to the originating UE (e.g., UE-A), a message comprising the avatar representation of the user.
[0346] At step 12, the originating UE (e.g., UE-A) may perform rendering.
[0347] The originating UE (e.g., UE-A) may transform video media comprising a video of a user into video media comprising a video of the avatar representation of the user using the video codec identified by the codec identifier.
[0348] Alternatively, the originating UE (e.g., UE-A) may transform the video media comprising the video of the user into avatar media comprising gestures of the user withinthe video of the user and the avatar representation of the user using the avatar codec identified by the codec identifier.
[0349] At step 13, the originating UE (e.g., IIE-A) may transmit to the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, the video media comprising the video of the avatar representation of the user or the avatar media comprising gestures of the user within the video of the user and the avatar representation of the user. The video media or the avatar media may be transmitted using RTP.
[0350] At step 14, the originating UE (e.g., UE-A) may receive, from the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, video media comprising a video of an avatar representation of another user. The video media may be received using RTP.
[0351] Alternatively, the originating UE (e.g., UE-A) may receive, from the terminating UE (e.g., UE-B) via the originating and terminating IMS networks, avatar media comprising gestures of another user within a video of the other user and the avatar representation of the other user. The avatar media may be received using RTP.
[0352] The originating UE (e.g., UE-A) may decode the video media comprising a video of the avatar representation of the other user using a video codec. The originating UE (e.g., UE-A) may display the video of the avatar representation of the other user.
[0353] Alternatively, the originating UE (e.g., UE-A) may decode the avatar media comprising avatar media comprising gestures of the other user within the video of the other user and the avatar representation of the other user. The originating UE (e.g., UE-A) may animate the avatar representation of the other user using the gestures of the other user to obtain a video of the avatar representation of the other user. The originating UE (e.g., UE- A) may display the video of the avatar representation of the other user.
[0354] Fig. 11 shows a block diagram of an originating user equipment based rendering for an avatar call between an originating user equipment (e.g., UE-A) and a terminating user equipment (e.g., UE-B).
[0355] At step 1100, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B), a message to establish or modify an IMS session between the originating UE (e.g., UE-A) and the terminating UE (e.g., UE-B) for an avatar call.
[0356] At step 1002, the originating UE (e.g., IIE-A) may generate first video media comprising a video of a user.
[0357] At step 1004, the originating UE (e.g., UE-A) may transform the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user.
[0358] At step 1006, the originating UE (e.g., UE-A) may transmit to the terminating UE (e.g., UE-B), the second video media or the avatar media.
[0359] Fig 12 shows a block diagram of a network based rendering for an avatar call between an originating user equipment (e.g., UE-A) and a terminating user equipment (e.g., UE-B).
[0360] At step 1200, the MF or MRF may receive, from the originating UE (e.g., UE-A), first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user.
[0361] At step 1202, the MF or MRF may transform the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar media comprising gestures of the user within the video of the user and an avatar representation of the user.
[0362] At step 1204, the MF or MRF may transmit to the terminating UE (e.g., UE-B), the second video media or the second avatar media.
[0363] Fig. 13 shows a block diagram of a terminating user equipment based rendering for an avatar call between an originating user equipment (e.g., UE-A) and a terminating user equipment (e.g., UE-B).
[0364] At step 1300, the terminating UE (e.g., UE-B) may receive, from the originating user equipment (e.g., UE-A), a message to establish or modify an IMS session between the originating user equipment (e.g., UE-A) and the terminating UE (e.g., UE-B) for an avatar call
[0365] At step 1302, the terminating UE (e.g., IIE-B) may receive, from the originating user equipment (e.g., IIE-A), avatar media comprising gestures of the user within a video of the user.
[0366] At step 1304, the terminating UE (e.g., UE-B) may decode the avatar media to obtain the gestures of the user within a video of the user.
[0367] At step 1306, the terminating UE (e.g., UE-B) may animate the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user.
[0368] At step 1306, the terminating UE (e.g., UE-B) may display the video of the avatar representation of the user.
[0369] Fig. 14 shows a schematic representation of non-volatile memory media 1400 storing instructions which when executed by a processor cause the processor to perform any one of the methods described in Fig. 11 , Fig. 12 and Fig. 13.
[0370] It is noted that while the above describes example embodiments, there are several variations and modifications which may be made to the disclosed solution without departing from the scope of the present invention.
[0371] It will be understood that although the above concepts have been discussed in the context of 5G, one or more of these concepts may be applied to other generations.
[0372] The embodiments may thus vary within the scope of the attached claims. In general, some embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although embodiments are not limited thereto. While various embodiments may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
[0373] The embodiments may be implemented by computer software stored in a memory and executable by at least one data processor of the involved entities or by hardware, or bya combination of software and hardware. Further in this regard it should be noted that any procedures, e.g., as in Fig. 11 , Fig. 12 and Fig. 13, may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD.
[0374] The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), gate level circuits and processors based on multi-core processor architecture, as non-limiting examples.
[0375] Alternatively or additionally some embodiments may be implemented using circuitry. The circuitry may be configured to perform one or more of the functions and / or method steps previously described. The circuitry may be provided in a base station and / or in a user equipment.
[0376] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuits (such as only analogue and / or digital circuits);(b) combinations of hardware circuits and software, such as:(i) a combination of analogue and / or digital hardware circuit(s) with software / firmware and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as the communications device or base station to perform the various functions previously described; and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) foroperation, but the software may not be present when it is not needed for operation.
[0377] This definition of circuitry applies to all uses of this term “means” in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example integrated device.
[0378] The foregoing description has provided by way of exemplary and non-limiting examples a full and informative description of some embodiments However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings will still fall within the scope as defined in the appended claims.
Claims
CLAIMS1 . A method of a first user equipment, the method comprising: transmitting, to a second user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call; generating first video media comprising a video of a user; transforming the first video media comprising the video of the user into second video media comprising a video of an avatar representation of the user or into avatar media comprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to the second user equipment, the second video media or the avatar media.
2. The method of claim 1 , wherein transforming the first video media into the second video media comprises: decoding the first video media to obtain the video of the user using a video codec; replacing the user in the video of the user with the avatar representation of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user to obtain the second video media using the video codec.
3. The method of claim 1 , wherein transforming the first video media into avatar media comprises: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user using an avatar codec to obtain the avatar media.
4. The method of any of claims 1 to 3, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call comprises a codec identifier identifying a video codec or an avatar codec; and wherein the method comprises: receiving, from the second user equipment, a message indicating acceptance of the codec identifier; and transforming the first video media into the second video media or into the avatar media using the video codec or the avatar codec identified by the codec identifier.
5. The method of any of claims 1 to 4, wherein the message to establish or modify an internet protocol multimedia subsystem session for an avatar call between the first user equipment and the second user equipment comprises an avatar identifier identifying the avatar representation of the user; and wherein the method comprises: receiving, from the second user equipment, a message indicating an acceptance of the avatar identifier; and transforming the first video media into the second video media or the avatar media using the avatar representation identified by the avatar identifier.
6. The method of claim 5, wherein the avatar identifier comprises an integer, a string, a universal unique identifier, a uniform resource name or a uniform resource locator.
7. The method of any of claims 1 to 6, comprising: retrieving the avatar representation of the user identified by the avatar identifier from a memory of the first user equipment.
8. The method of any of claims 1 to 7, comprising: receiving, from a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user identified by the avatar identifier.
9. The method of claim 8, wherein the avatar representation of the user identified by the avatar identifier is received using an internet protocol multimedia subsystem data channel.
10. The method of any of claims 1 to 9, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call is transmitted using an internet protocol multimedia subsystem data channel or using a hypertext transfer protocol.
11. The method of any of claims 1 to 10, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call is transmitted using a session description protocol.
12. The method of any of claims 1 to 11 , wherein the second video media or the avatar media is transmitted to the second user equipment using a real time protocol.
13. A method of an internet protocol multimedia subsystem network, the method comprising: receiving, from a first user equipment, first video media comprising a video of a user or first avatar media comprising gestures of the user within a video of the user; transforming the first video media comprising the video of the user or the first avatar media comprising gestures of the user within the video of the user into second video media comprising a video of an avatar representation of the user or into second avatar mediacomprising gestures of the user within the video of the user and an avatar representation of the user; and transmitting, to a second user equipment, the second video media or the second avatar media.
14. The method of claim 13 comprising: receiving the avatar representation of the user from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem.
15. The method of claim 13 or claim 14, wherein the avatar representation is received using an internet protocol multimedia subsystem data channel or a hypertext transfer protocol.
16. The method of any of claims 13 to 15, wherein transforming the first video media into the second video media comprises: decoding the first video media to obtain the video of the user using a video codec; replacing the user with the avatar representation of the user in the video of the user to obtain the video of the avatar representation of the user; and encoding the video of the avatar representation of the user using the video codec to obtain the second video media.
17. The method of any of claims 13 to 15, wherein transforming the first avatar media into the second video media comprises: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; animating the avatar representation of the user using the gestures of the user within the video of the user to obtain the video of the avatar representation of the user; andencoding the video of the avatar representation of the user using the video codec to obtain the second video media.
18. The method of any of claims 13 to 15, wherein transforming the first video media into the second avatar media comprises: decoding the first video media to obtain the video of the user using a video codec; determining gestures of the user within the video of the user; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using an avatar codec.
19. The method of any of claims 13 to 15, wherein transforming the first avatar media into the second avatar media comprises: decoding the first avatar media comprising gestures of the user within a video of the user using an avatar codec; and encoding the gestures of the user and the avatar representation of the user to obtain the second avatar media using the avatar codec.
20. The method of any of claims 13 to 19, comprising: receiving, from a network function part of the internet protocol multimedia subsystem network or a network function not part the internet protocol multimedia subsystem network, a message indicating a codec identifier identifying a video codec or an avatar codec; and transforming the first video media or the first avatar media into the second video media or the second avatar media using the video codec or the avatar codec identified by the codec identifier.21 . The method of any of claims 13 to 20, comprising: receiving, from the first user equipment, from a network function part of the internet protocol multimedia subsystem network or from a network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user; and transforming the first video media or first avatar media into the second video media or the second avatar media using the avatar representation.
22. The method of any of claims 13 to 21 , comprising: transmitting, to the first user equipment, the second video media or the second avatar media.
23. The method of any of claims 13 to 22, wherein the second video media or the second avatar media is transmitted to at least one of the first user equipment or the second user equipment using a real time protocol.
24. The method of any of claims 13 to 23, wherein the gestures of the user within the video of the user and the avatar representation of the user are transmitted via an internet protocol multimedia subsystem data channel.
25. The method of any of claims 13 to 24, where the method is performed by a media function or a media resource function.
26. A method of a second user equipment, the method comprising: receiving, from a first user equipment, a message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call;receiving, from the first user equipment, avatar media comprising gestures of the user within a video of the user; decoding the avatar media to obtain the gestures of the user within a video of the user; animating the avatar representation using the gestures of the user within the video of the user to obtain a video of the avatar representation of the user; and displaying the video of the avatar representation of the user.
27. The message of claim 26, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call comprises a codec identifier identifying an avatar codec; and wherein the method comprises: transmitting, to the first user equipment, a message indicating acceptance of the codec identifier; and decoding the avatar media to obtain the gestures of the user within a video of the user using the avatar codec identified by the codec identifying.
28. The method of claim 26 or claim 27, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call comprises an avatar identifier identifying an avatar representation of the user; and wherein the method comprises: transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; transmitting, to a network function part of an internet protocol multimedia subsystem network or a network function not part of the internet protocol multimedia subsystemnetwork, a message comprising the avatar identifier identifying the avatar representation of the user; and receiving, from the network function part of the internet protocol multimedia subsystem network or the network function not part of the internet protocol multimedia subsystem network, the avatar representation of the user.
29. The method of claim 26 or claim 27, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call comprises an avatar identifier identifying an avatar representation of the user; and wherein the method comprises: transmitting, to the first user equipment, a message indicating acceptance of the avatar identifier; and retrieving the avatar representation of the user identified by the avatar identifier from a memory of the second user equipment.
30. The method of any claims 26 to 29, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call is transmitted using an internet protocol multimedia subsystem data channel.
31. The method of any of claims 26 to 30, wherein the message to establish or modify an internet protocol multimedia subsystem session between the first user equipment and the second user equipment for an avatar call is transmitted using a session description protocol.
32. The method of any of claims 26 to 31 , wherein the avatar media is received from the first user equipment using a real time protocol.
33. The method of any of claims 26 to 32, wherein the avatar media comprising gestures of the user within a video of the user is received via an internet protocol multimedia subsystem data channel.
34. A first user equipment, comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, causes the first user equipment to carry out the method of any of claims 1 to 12.
35. An internet protocol multimedia subsystem network, comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, causes the internet protocol multimedia subsystem network to carry out the method of any of claims 13 to 25.
36. A second user equipment, comprising: at least one processor; and at least one memory storing instructions which, when executed by the at least one processor, causes the second user equipment to carry out the method of any of claims 26 to 33.
37. A computer program comprising computer executable code which when run on at least one processor of a user equipment causes the user equipment to perform the method of any of claims 1 to 12 or 26 to 33.
38. A computer program comprising computer executable code which when run on at least one processor of an internet protocol multimedia subsystem network causes the internet protocol multimedia subsystem network to perform the method of any of claims 13 to 25.
39. A computer-readable medium comprising instructions which, when executed by a user equipment, causes the user equipment to perform the method of any of claims 1 to 12 or 26 to 33.
40. A computer-readable medium comprising instructions which, when executed by at least one processor of an internet protocol multimedia subsystem network, causes the internet protocol multimedia subsystem network to perform the method of any of claims 13 to 25.
Citation Information
Patent Citations
Avatar call platform
US20230199147A1