Augmented reality communication method, device, and system

Abstract

In order to provide an implementation in which AR is integrated into voice and video calls, thereby improving user experience, an augmented reality communication method, device, and system are provided. An auxiliary data channel and a media stream channel are established between a terminal device and an AR media processing network element so that AR can be effectively integrated into voice and video calls. Also, since the AR auxiliary data and the media stream have the same transmission path, the latency can be reduced. The AR media processing network element may be an IMS access gateway with AR processing capability or a media resource function MRF.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Chinese Patent Application No. 202110329252.9, entitled “Augmented Reality Communication Method, Apparatus, and System,” filed with the China State Intellectual Property Office on March 27, 2021, which is incorporated herein by reference in its entirety.

[0002] TECHNICAL FIELD Embodiments of this application relate to the field of communication technologies, and in particular to an augmented reality communication method, device, and system. [Background technology]

[0003] Built on the 4th generation (4G) network, voice over long term evolution (VoLTE) is an end-to-end voice solution under all-IP conditions. During communication between users, VoLTE provides shorter call connection time and higher voice and video call quality. Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. Multiple technical means, such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing, are widely used to simulate virtual information, such as computer-generated text, images, three-dimensional models, music, and videos. Then, pseudo-information is applied to the real world to complement the two types of information, thereby implementing the "augmentation" of the real world. Augmented reality technology can not only effectively reflect the content of the real world, but also display the content of virtual information content. Currently, there is no effective implementation in which AR is integrated into voice and video calls. Summary of the Invention

[0004] Embodiments of this application provide an augmented reality communication method, apparatus, and system for providing an implementation in which AR is integrated into voice and video calls, thereby improving the user experience. [Means for solving the problem]

[0005] According to a first aspect, an embodiment of this application provides an augmented reality AR communication system, including an Internet Protocol Multimedia Subsystem (IMS) core network element and a first AR media processing network element. The IMS core network element is configured to control the establishment of a first media stream channel between the first AR media processing network element and a first terminal device, and an auxiliary data channel between the first AR media processing network element and the first terminal device. The first AR media processing network element is configured to receive first AR auxiliary data from the first terminal device through the established auxiliary data channel, and receive a first media stream for AR communication between the first terminal device and a second terminal device through the established media stream channel. The first AR media processing network element is further configured to perform enhanced processing on the media stream based on the first AR auxiliary data to obtain a first AR media stream, and transmit the first AR media stream to the first terminal device through the media stream channel. According to the aforementioned solution, the auxiliary data channel and the media stream channel are established between the terminal device and the AR media processing network element, so that AR is effectively integrated into voice and video calls. Also, since the AR auxiliary data and the media stream have the same transmission path, the latency can be reduced.

[0006] In a possible design, the first AR media processing network element may be an IMS access gateway having an AR processing capability or a media resource function MRF. For example, when point-to-point communication is performed between terminal devices, the first AR media processing network element may be an IMS access gateway. For another example, when terminal devices perform an AR conference, the first AR media processing network element may be an MRF.

[0007] In a possible design, the first AR media processing network element is located in the IMS network to which the first terminal device belongs (or in which the first terminal device is located). The AR media processing network element in the IMS network in which the first terminal device is located has AR processing capability, and the AR media processing network element in the IMS network in which the first terminal device is located performs enhanced processing according to the proximity principle, so that the latency can be further reduced.

[0008] In a possible design, the system further includes a second AR media processing network element located in an IMS network to which the second terminal device belongs (or in which the second terminal device is located). The first AR media processing network element is further configured to transmit the AR media stream to the second AR media processing network element. The second AR media processing network element is configured to receive the AR media stream transmitted by the first AR media processing network element and transmit the AR media stream to the second terminal device.

[0009] In a possible design, the first AR media processing network element is located in an IMS network to which the second terminal device belongs (or in which the second terminal device is located), and the system further includes a third AR media processing network element located in the IMS network to which the first terminal device belongs (or in which the first terminal device is located), where the third AR media processing network element does not have AR processing capability, and the media stream channel between the first AR media processing network element and the first terminal device includes a first media stream channel between the first AR media processing network element and the third AR media processing network element and a second media stream channel between the third AR media processing network element and the first terminal device.

[0010] The third AR media processing network element is configured to receive a media stream from the first terminal device through the second media stream channel and forward the media stream to the first AR media processing network element through the first media stream channel, and to receive an AR media stream from the first AR media processing network element through the first media stream channel and transmit the AR media stream to the first terminal device through the second media stream channel.

[0011] In the above design, if an AR media processing network element in an IMS network in which a first terminal device is located does not have AR processing capability, an AR media processing network element in an IMS network in which a second terminal device is located is configured to provide the first terminal device with enhanced processing capabilities. A feasible solution for incorporating AR into voice and video calls is provided.

[0012] In a possible design, the first AR media processing network element is further configured to receive target object information from the first terminal device over an auxiliary data channel. The AR media processing network element is further configured to obtain a target object corresponding to the target object information from a third party server.

[0013] In such designs, the third-party server may provide material of the target object for AR communication between the terminal devices. In this application, the target object is obtained from the third-party server through an auxiliary data channel.

[0014] In a possible design, the target object is a virtual object or an identifier of the virtual object. The first AR media processing network element is further configured to send the virtual object or an identifier of the virtual object to the first terminal device through an auxiliary data channel. Because the virtual object is required in multiple scenarios, in such a design, the virtual object is obtained from a third-party server through the auxiliary data channel.

[0015] In a possible design, the target object information includes a virtual object type requested by the first terminal device, and the first AR media processing network element sends multiple virtual object identifiers of the virtual object type to the first terminal device over an auxiliary data channel.

[0016] In a possible design, after receiving the multiple virtual object identifiers corresponding to the virtual objects, the first AR media processing network element stores a correspondence between the virtual object types and the multiple virtual object identifiers.

[0017] In a possible design, when receiving the virtual object type through the auxiliary data channel, the first AR media processing network element may first determine whether the first AR media processing network element stores a virtual object identifier corresponding to the virtual object type. If so, the first AR media processing network element may not obtain a virtual object identifier corresponding to the virtual object type from a third-party server. If the first AR media processing network element does not store a virtual object identifier corresponding to the virtual object type, the first AR media processing network element obtains a virtual object identifier corresponding to the virtual object type from a third-party server. In this way, unnecessary resource waste caused by the acquisition can be reduced.

[0018] In one possible design, the method further includes:

[0019] The first AR media processing network element receives a first virtual object identifier from the first terminal device over the auxiliary data channel, the first virtual object identifier for identifying a first virtual object selected by a user of the first terminal device from the plurality of virtual objects. The first AR media processing network element retrieves the first virtual object corresponding to the first virtual object identifier from the third party server and transmits the first virtual object to the first terminal device over the auxiliary data channel.

[0020] In a possible design, the target object is an actual background object, and the first AR media processing network element is further configured to receive, via a media stream channel, a second media stream generated during the AR communication and transmitted by the first terminal device.

[0021] The first AR media processing network element is further configured to render and combine the actual background object and the second media stream to obtain a second AR media stream, and transmit the second AR media stream to the first terminal device through a media stream channel. In some application scenarios, the background picture corresponding to the foreground object is fixed. In this scenario, the rendering operation between the media stream and the background picture is performed by the AR media processing network element, so that the pressure on the uplink bandwidth of the terminal device can be reduced.

[0022] In a possible design, the system further includes an application server.

[0023] The application server is configured to receive a first request message from the AR media processing network element and forward the first request message to the third party server, and to receive a first response message from the third party server and forward the first response message to the first AR media processing network element.

[0024] In a possible design, the application server is configured with an address of the third party server, and the application server is specifically configured to forward the first request message to the third party server based on the address of the third party server and to forward the first response message to the first AR media processing network element.

[0025] In a possible design, the application server is further configured to send the address of the third party server to the first AR media processing network element in a procedure for establishing an auxiliary data channel between the first terminal device and the first AR media processing network element.

[0026] According to a second aspect, an embodiment of the present application provides an augmented reality AR communication method, the method including: an Internet Protocol Multimedia Subsystem (IMS) core network element receiving a first request message from a first terminal device, the first request message being for requesting establishment of an auxiliary data channel, the auxiliary data channel being for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element. The IMS core network element controls the establishment of the auxiliary data channel between the AR media processing network element and the first terminal device. According to the aforementioned solution, a simple and effective manner of creating an auxiliary data channel is provided for obtaining AR auxiliary data for AR communication through the auxiliary data channel and performing enhanced processing on a media stream for AR communication during the AR communication.

[0027] In a possible design, the IMS core network element controlling establishment of the auxiliary data channel between the AR media processing network element and the first terminal device includes the IMS core network element sending a second request message to the AR media processing network element, the second request message for requesting establishment of the auxiliary data channel between the AR media processing network element and the first terminal device. The IMS core network element receives a second response message sent by the AR media processing network element, the second response message indicating to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel. The IMS core network element sends a first response message to the first terminal device, the first response message indicating to the first terminal device that the AR media processing network element is ready to establish the auxiliary data channel.

[0028] In a possible design, the AR media processing network element is an IMS access gateway having an AR processing capability or a media resource function MRF. For example, when point-to-point communication is performed between terminal devices, the first AR media processing network element may be an IMS access gateway. In another example, when an AR conference is performed between terminal devices, the first AR media processing network element may be an MRF.

[0029] In a possible design, the IMS core network element controls the establishment of a media stream channel between the AR media processing network element and the first terminal device. According to the above solution, an auxiliary data channel and a media stream are established between the IMS core network element and the AR media processing network element, so that AR is effectively integrated into voice and video calls. And since the AR auxiliary data and the media stream have the same transmission path, the latency can be reduced.

[0030] In a possible design, the AR media processing network element is located in the IMS network to which the first terminal device belongs, or is located in the IMS network to which the second terminal device belongs. The AR media processing network element in the IMS network to which the first terminal device is located has AR processing capability, and the AR media processing network element in the IMS network to which the first terminal device is located performs enhanced processing according to the proximity principle, so that the latency can be further reduced. If the AR media processing network element in the IMS network to which the first terminal device is located does not have AR processing capability, the AR media processing network element in the IMS network to which the second terminal device is located is configured to provide the first terminal device with enhanced processing capabilities. A feasible solution for incorporating AR into voice and video calls is provided.

[0031] In a possible design, the first request message and the second request message carry first descriptive parameters of the first terminal device for establishing the auxiliary data channel.

[0032] The first response message and the second response message carry second description parameters of the AR media processing network element for establishing the auxiliary data channel.

[0033] In a possible design, the method further includes the IMS core network element sending a query request to the AR media processing network element, where the query request is for querying whether the AR media processing network element has AR processing capability. The IMS core network element receives the query response sent by the AR media processing network element, where the query response carries information indicating that the AR media processing network element has AR processing capability. With the above design, a feasible and effective manner of determining whether the AR media processing network element has AR processing capability is provided.

[0034] In a possible design, the AR media processing network element is located in an IMS network to which the second terminal device belongs, and the method further includes:

[0035] Before sending an inquiry message to an AR media processing network element located in an IMS network to which the second terminal device belongs, the IMS core network element determines that the AR media processing network element serving the first terminal device does not have AR processing capability.

[0036] In a possible design, the procedure for establishing AR communication between the first terminal device and the second terminal device is a procedure for establishing an AR session between the first terminal device and the second terminal device, a procedure for updating the AR session between the first terminal device and the second terminal device, or a procedure for re-establishing the AR session between the first terminal device and the second terminal device.

[0037] In a possible design, the first request message is a Session Initiation Protocol (SIP) request INVITE message, or a SIP Re-establish Request message, or a SIP Update Request message.

[0038] In a possible design, the method further includes the IMS core network element controlling establishment of a media stream channel between the AR media processing network element and the first terminal device, the media stream channel for transmitting a media stream for transmitting content for AR communication between the first terminal device and the AR media processing network element.

[0039] In a possible design, a third request message is received from the first terminal device, the third request message further for requesting establishment of the media stream channel. The IMS core network element controlling the establishment of the media stream channel between the AR media processing network element and the first terminal device includes the IMS core network element sending a fourth request message to the AR media processing network element, the fourth request message for requesting establishment of the media stream channel between the AR media processing network element and the first terminal device. The IMS core network element receives a fourth response message sent by the AR media processing network element, the fourth response message indicating to the IMS core network element that the AR media processing network element agrees to the establishment of the media stream channel. The IMS core network element sends a third response message to the first terminal device, the third response message indicating to the first terminal device that the AR media processing network element is ready to establish the media stream channel.

[0040] It should be noted that the auxiliary data channel and the media stream channel may be established in the same procedure, or the media stream channel may be established first and the auxiliary data channel then established.

[0041] For example, the first request message and the third request message may be transmitted in the same message, the second request message and the fourth request message may be transmitted in the same message, the first response message and the third response message may be transmitted in the same message, and the second response message and the fourth response message may be transmitted in the same message.

[0042] According to a third aspect, an embodiment of the present application provides an augmented reality (AR) communication method, the method including:

[0043] An auxiliary data channel is established between the AR media processing network element and the first terminal device under the trigger of the first terminal device.

[0044] In a possible design, a media stream channel for transmitting content for AR communication is established between the AR media processing network element and the first terminal device.

[0045] In a possible design, the AR media processing network element receives a second request message sent by an Internet Protocol Multimedia Subsystem (IMS) core network element, where the second request message is for requesting establishment of an auxiliary data channel between the AR media processing network element and a first terminal device, where the auxiliary data channel is for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element. The AR media processing network element sends a second response message to the IMS core network element, where the second response message indicates to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel.

[0046] In a possible design, the method further includes, after establishment of the auxiliary data channel is completed, the AR media processing network element receiving AR auxiliary data from the first terminal device over the auxiliary data channel.

[0047] In a possible design, the method further includes the AR media processing network element receiving a media stream for the AR communication from the first terminal device through a media stream channel. The AR media processing network element performs media enhancement processing on the media stream for the AR communication based on the AR auxiliary data to obtain the AR media stream. The AR media processing network element transmits the AR media stream obtained by the enhancement processing to the first terminal device through the media stream channel. According to the aforementioned solution, the auxiliary data channel and the media stream are established between the terminal device and the AR media processing network element so that the AR is effectively integrated into the voice call and the video call. Also, since the AR auxiliary data and the media stream have the same transmission path, the latency can be reduced.

[0048] In a possible design, the method further includes the AR media processing network element transmitting the AR media stream obtained by the enhanced processing to the second terminal device.

[0049] In possible designs, the AR auxiliary data includes one or more of virtual model manipulation data, target object annotation or tracking data, facial expression or body action identification data, or pose data.

[0050] In a possible design, the AR media processing network element is an IMS access gateway with AR processing capability or a media resource function MRF.

[0051] In a possible design, the AR media processing network element is located in an IMS network to which the first terminal device belongs (or in which the first terminal device is located), or is located in an IMS network to which the second terminal device belongs (or in which the second terminal device is located).

[0052] In a possible design, the second request message carries first description parameters for the first terminal device for establishing an auxiliary data channel, and the method further includes the AR media processing network element determining from the first description parameters second description parameters supported by the AR media processing network element, and the second response message carries the second description parameters.

[0053] According to a fourth aspect, an embodiment of the present application provides an augmented reality (AR) communication method, the method including: a first terminal device sending a first request message to an IMS core network element, the first request message for requesting establishment of an auxiliary data channel. The first terminal device receives a first response message sent by the IMS core network element, the first response message indicating that the AR media processing network element completes the establishment of the auxiliary data channel, the auxiliary data channel for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element.

[0054] In a possible design, the method further includes establishing a media stream channel between the first terminal device and the AR media processing network element for transmitting content for the AR communication.

[0055] In a possible design, establishing a media stream channel for transmitting content for AR communication between the first terminal device and the AR media processing network element includes:

[0056] The first terminal device sends a third request message to the IMS core network element, where the third request message is to request the establishment of a media stream channel.

[0057] The first terminal device receives a third response message sent by the IMS core network element, where the third response message indicates to the first terminal device that the AR media processing network element is ready to establish a media stream channel.

[0058] In a possible design, the first request message is for further requesting the establishment of a media stream channel, and the first response message further indicates that the AR media processing network element is ready to establish the media stream channel.

[0059] In one possible design, the method further includes:

[0060] The first terminal device transmits augmented reality AR auxiliary data to the AR media processing network element through an auxiliary data channel and transmits a media stream to the AR media processing network element through a media stream channel, the media stream being generated during AR communication between the first terminal device and the second terminal device, and the AR auxiliary data being used by the AR media processing network element to perform media enhancement processing on the media stream for the AR communication.

[0061] The first terminal device receives the AR media stream obtained by the enhanced processing from the AR media processing network element through a media stream channel.

[0062] In one possible design, the method further includes:

[0063] The first terminal device receives, via the media stream channel, an AR media stream belonging to a second terminal device and transmitted by the AR media processing network element.

[0064] In a possible design, the first request message carries first descriptive parameters of the first terminal device for establishing the auxiliary data channel.

[0065] The first response message carries second description parameters of the AR media processing network element for establishing the auxiliary data channel.

[0066] In a possible design, the AR media processing network element is located in an IMS network to which the first terminal device belongs (or in which the first terminal device is located), or is located in an IMS network to which the second terminal device belongs (or in which the second terminal device is located).

[0067] According to a fifth aspect, an embodiment of the present application further provides an apparatus. The apparatus is used in an IMS core network element. For beneficial effects, please refer to the description of the second aspect. Details will not be repeated in this specification. The apparatus has a function of implementing the behavior in the method example of the second aspect. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. In a possible design, the structure of the apparatus may include a transmitting unit and a receiving unit. These units can perform corresponding functions in the method example of the second aspect. For details, please refer to the detailed description in the method example. Details will not be repeated in this specification.

[0068] According to a sixth aspect, an embodiment of this application further provides an apparatus. The apparatus is used in an AR media processing network element. For beneficial effects, please refer to the description of the third aspect. Details will not be repeated in this specification. The apparatus has a function of implementing the behavior in the method example of the third aspect. This function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. In a possible design, the structure of the apparatus may include a transmitting unit, a receiving unit, and a processing unit. The units can perform corresponding functions in the method example of the third aspect. For details, please refer to the detailed description in the method example. Details will not be repeated in this specification.

[0069] According to a seventh aspect, an embodiment of the present application further provides an apparatus. The apparatus is used in a terminal device. For beneficial effects, please refer to the description of the fourth aspect. Details will not be repeated in this specification. The apparatus has a function of implementing the behavior in the method example of the fourth aspect. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. In a possible design, the structure of the apparatus includes a transmitting unit and a receiving unit. The units can perform corresponding functions in the method example of the fourth aspect. For details, please refer to the detailed description in the method example. Details will not be repeated in this specification.

[0070] According to an eighth aspect, an embodiment of this application further provides an apparatus. The apparatus is used in an IMS core network element. For beneficial effects, please refer to the description of the second aspect. Details will not be repeated in this specification. The structure of the communication apparatus includes a processor and a communication interface, and may further include a memory. The processor is configured to support the IMS core network element in performing corresponding functions in the method of the second aspect through the communication interface. The memory is coupled to the processor and stores program instructions and data required for the communication apparatus.

[0071] According to a ninth aspect, an embodiment of this application further provides an apparatus. The apparatus is used in an AR media processing network element. For beneficial effects, please refer to the description of the third aspect. Details will not be described repeatedly in this specification. The structure of the apparatus includes a processor and a communication interface, and may further include a memory. The processor is configured to support the AR media processing network element in performing corresponding functions in the method of the third aspect through the communication interface. The memory is coupled to the processor and stores program instructions and data required for the apparatus.

[0072] According to a tenth aspect, an embodiment of this application further provides an apparatus. The apparatus is used in a terminal device. For beneficial effects, please refer to the description of the fourth aspect. Details will not be repeated in this specification. The structure of the apparatus includes a processor and a communication interface, and may further include a memory. The processor is configured to support the terminal device in performing corresponding functions in the method of the fourth aspect through the communication interface. The memory is coupled to the processor and stores program instructions and data required for the apparatus.

[0073] According to an eleventh aspect, an embodiment of the present application further provides an apparatus, the apparatus being used in an application server. The structure of the communication apparatus includes a processor and a communication interface, and may further include a memory. The processor is configured to support the application server in performing the corresponding functions of the application server in any one of the aforementioned aspects through the communication interface. The memory is coupled to the processor and stores program instructions and data required for the communication apparatus.

[0074] According to a twelfth aspect, the application further provides a computer-readable storage medium storing instructions which, when executed on a computer, enable the computer to perform a method according to the previous aspect.

[0075] According to a thirteenth aspect, the application further provides a computer program product comprising instructions, which when run on a computer, enable the computer to perform a method according to the previous aspect.

[0076] According to a fourteenth aspect, the present application further provides a computer chip, the chip being connected to a memory, the chip being configured to read and execute a software program stored in the memory to perform a method according to the previous aspect. [Brief description of the drawings]

[0077] [Figure 1] FIG. 1 is a schematic diagram of a possible architecture of an AR communication system according to one embodiment of this application. [Figure 2A] FIG. 2 is a schematic diagram of another possible architecture of an AR communication system according to an embodiment of the present application. [Figure 2B] FIG. 1 is a schematic diagram of yet another possible architecture of an AR communication system according to an embodiment of the present application. [Figure 2C]FIG. 1 is a schematic diagram of yet another possible architecture of an AR communication system according to an embodiment of the present application. [Diagram 3] FIG. 2 is a schematic diagram of another possible architecture of an AR communication system according to an embodiment of the present application. [Figure 4] FIG. 2 is a schematic diagram of another possible architecture of an AR communication system according to an embodiment of the present application. [Diagram 5] FIG. 2 is a schematic diagram of another possible architecture of an AR communication system according to an embodiment of the present application. [Figure 6] 1 is a schematic flowchart of an AR communication method according to an embodiment of the present application. [Figure 7A] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 7B] 1 is a schematic flowchart of yet another AR communication method according to an embodiment of the present application. [Figure 8A-1] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 8A-2] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 8B] FIG. 2 is a schematic diagram of a termination T1 and a termination T2 on an IMS AGW according to an embodiment of the present application. [Figure 9] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 10A] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 10B] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 11] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 12A] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 12B] 1 is a schematic flowchart of yet another AR communication method according to an embodiment of the present application. [Figure 12C] 1 is a schematic flowchart of yet another AR communication method according to an embodiment of the present application. [Figure 13] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 14] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 15] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 16] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 17] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 18] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 19] 1 is a schematic flowchart of another AR communication method according to an embodiment of the present application. [Figure 20] FIG. 1 is a schematic diagram of the structure of an apparatus according to one embodiment of the present application. [Figure 21] FIG. 2 is a schematic diagram of the structure of another device according to an embodiment of the present application. [Figure 22] FIG. 2 is a schematic diagram of the structure of another device according to an embodiment of the present application. [Figure 23] FIG. 2 is a schematic diagram of the structure of another device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0078] This application provides an AR communication method and device to provide an implementation that improves user experience by incorporating AR into voice and video calls. The voice and video calls may be, but are not limited to, VoLTE, and may also be applicable to voice and video calls provided by future technologies. AR has three characteristics: virtual reality integration, real-time interaction, and three-dimensional registration. Virtual reality integration is the superimposition of computer-generated virtual objects and information onto real-world scenes to provide a more intuitive and deeper understanding of the real scene. The augmented information may be non-geometric information related to real objects, such as video and text, or geometric information, such as virtual three-dimensional objects and scenes. The real-time interaction is the real-time interaction between people and the augmented reality environment in a natural way via a dialogue interface device in the augmented reality system. The "registration" in three-dimensional registration can be interpreted as tracking and localization. The three-dimensional registration is that the computer-generated virtual objects correspond one-to-one with the real environment, and the correct registration relationship is continuously maintained as the user moves in the real environment.

[0079] The term "and / or" in this application is limited to describing a relationship of association to describe related things, and indicates that three relationships may exist. For example, A and / or B represents the following three cases: when only A is present, when both A and B are present, and when only B is present. In addition, the symbol " / " in this specification generally indicates an "or" relationship between related objects. The term "at least one" in this application means one or more, i.e., including one, two, three, or more, and the term "plurality" means two or more, i.e., including two, three, or more. In addition, it can be understood that in the description of this application, terms such as "first" and "second" are merely used for distinction and explanation, but cannot be understood as indicating or implying a relative importance, or as indicating or implying an order. The term "at least one of the following items" or similar expressions refers to any combination of these items, including any combination of a single item or multiple items. For example, at least one of a, b, or c may refer to a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural. In the embodiments of this application, "B corresponding to A" may be understood to indicate that B is associated with A and B may be determined according to A. However, it may be further understood that determining B based on A does not mean that B is determined based only on A. B may alternatively be determined based on A and / or other information. In addition, the terms "include" and "have" in the embodiments, claims, and accompanying drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the enumerated steps or modules, and may further include unenumerated steps or modules.

[0080] Hereinafter, the solutions provided in the embodiments of this application will be described in detail with reference to the accompanying drawings.

[0081] FIG. 1 is a schematic diagram of an architecture of an AR communication system according to an embodiment of this application. The AR communication system includes an AR media processing network element and an AR control network element. The AR media processing network element provides media processing services to a terminal device having AR capability. The AR media processing network element has an AR User plane (AR-U) function and may be referred to as an AR-U network element. The AR control network element has an AR control plane function and may be referred to as an AR Control plane (AR-C) network element. In this embodiment, the network element may be referred to as a device. For example, the AR media processing network element may be referred to as an AR media processing device.

[0082] A terminal device with AR function may have one or more of the following logical functions: support for acquiring audio and video data in a real scene via a camera and / or a sensor, support for logical operation function (e.g., a visual engine may be deployed), support for low-latency encoding and lightweight image rendering, support for acquiring virtual objects from the cloud, etc. The terminal device in this embodiment of this application may be a device configured with a camera and having a video call function. For example, the terminal device may be a wearable device (e.g., an electronic watch), and the terminal device may be a device such as a mobile phone or a tablet computer. The specific form of the terminal device is not particularly limited in this embodiment of this application.

[0083] The AR media processing network element has an AR media processing function. For example, the AR media processing function includes one or more of three-dimensional modeling, object identification, tracking and localization, interaction perception, virtual scene rendering, virtual reality integration, etc. Three-dimensional modeling is to perform three-dimensional construction on the real environment based on data acquired by the terminal device. Object identification is to identify related objects of the terminal device user. Tracking and localization is to track and locate related objects of the terminal device user. Interaction perception is to implement multi-sensory consistency using user interaction techniques. Virtual scene rendering is to render a virtual scene. Virtual reality integration is to integrate virtual objects with real objects.

[0084] The AR media processing network element supports the establishment of a media stream channel and an auxiliary data channel with a terminal device. The media stream channel is for transmitting AR core data (or called media stream data, also called media stream for short), e.g., audio and video streams, between the terminal device and the AR media processing network element. The auxiliary data channel is for transmitting AR auxiliary data between the terminal device and the AR media processing network element. The AR auxiliary data (or AR specific data) is a general name for non-media data transmitted between the terminal device and the AR media processing network element.

[0085] In one example, as shown in Table 1, the data types that may be included in the AR auxiliary data in the uplink direction and the video data types that are included in the AR core data are described as examples.

[0086] [Table 1]

[0087] In one example, as shown in Table 2, the data types that may be included in the AR assistance data in the downlink direction and the video data types included in the audio stream and the video stream are described as an example.

[0088] [Table 2]

[0089] It can be understood that Tables 1 and 2 are used only as examples and do not constitute any particular limitations on the AR auxiliary data and AR core data.

[0090] The AR control network element has the functions of session association and service control. In an AR session scenario, the session association can include an association between a media stream between a terminal device and an AR media processing network element, and a signaling stream between an AR media processing network element and an AS, or includes an association between a media stream between a terminal device and an AR media processing network element, a signaling stream between an AR media processing network element and an AS, and a message stream between an AS and a third-party server. The third-party server is configured to provide materials required for AR processing, such as virtual objects and real background objects. The service control may be to control AR media processing based on service features, etc. For example, different service features correspond to different AR media processing methods, and the AR control network element can indicate different AR media processing methods to the AR media processing network element based on different service features of the terminal device.

[0091] In a possible implementation, the AR media processing network element may be an IP multimedia subsystem (IMS) access gateway (AGW) or a media resource function (MRF). The AR control network element may be an application server.

[0092] As an example, as shown in FIG. 2A, the AR media processing network element is an IMS AGW, and the AR communication system includes an IMS AGW and an IMS core. The IMS core controls the establishment of an auxiliary data channel and a media stream channel between the IMS AGW having AR processing capability and a terminal device having AR capability. The auxiliary data channel is for transmitting AR auxiliary data for AR communication between at least two terminal devices. The media stream channel is for transmitting media streams for AR communication between at least two terminal devices. The IMS AGW can provide a media processing service to the terminal device. For example, upon receiving a media stream through the media stream channel and receiving AR auxiliary data through the auxiliary data channel, the IMS AGW can perform enhanced processing on the media stream using the AR auxiliary data.

[0093] The AR communication system may further include an application server (AS). The AR communication system may further include at least two terminal devices. The IMS AGWs serving different terminal devices respectively may be the same or different. In FIG. 2A, for example, the two terminal devices are a first terminal device and a second terminal device respectively, and the IMS AGWs serving the first terminal device and the second terminal device are different, being IMS AGW1 and IMS AGW2 respectively. The two terminal devices in the AR communication may both have AR capability, or one terminal device may have AR capability and the other terminal device does not. In FIG. 2A, for example, both the first terminal device and the second terminal device have AR capability. For example, different terminal devices access different IMS networks, and the IMS AGWs serving different terminal devices are different. Different IMS AGWs serving two terminal devices in AR communication may both have AR processing capability, or only one IMS AGW has AR processing capability, and the other IMS AGW does not. In FIG. 2A, both IMS AGW1 and IMS AGW2 have AR processing capability. IMS AGW1 serves a first terminal device, or IMS AGW1 is an access gateway of the first terminal device, or IMS AGW1 is located in the IMS network to which the first terminal device belongs (or in which the first terminal device is located). IMS AGW2 serves a second terminal device. In other words, IMS AGW2 is an access gateway of the second terminal device, or IMS AGW2 is located in the IMS network to which the second terminal device belongs. One access gateway can serve multiple terminal devices. For example, IMS AGW1 provides a service to a first terminal device and further provides a service to a third terminal device (not shown in FIG. 2A).The IMS core may include a call session control function (CSCF) and / or a home subscriber server (HSS). The IMS core may further include another network element. In this embodiment of this application, details will not be described. The CSCF is a call control center of the IMS core, and implements functions such as user access, authentication, session routing, and service triggering on an IP transmission platform. The CSCF may include one or more of a serving-call session control function (S-CSCF), a proxy-CSCF (P-CSCF), and an interrogating-CSCF (I-CSCF). The HSS is configured to record subscription data of a user (such as user information and service data). It should be noted that an IMS core network element serving a first terminal device may be different from an IMS core network element serving a second terminal device, for example, related to the location of the terminal device. As an example, the IMS core network element referred to in this embodiment of this application may include a CSCF.

[0094] In some embodiments, the AR communication system may further include a third party server. The third party server may be understood as a server of a third party provider. The third party server is configured to provide target objects, such as virtual objects, or in another example, real background objects, required by the terminal device to perform the AR communication. The third party server and the application server are each configured with each other's address information, which may include, for example, an IP address and a port number. For a possible example, see FIG. 2B. The third party server supports the establishment of a communication connection to the AR media processing network element, which may be, for example, a hypertext transfer protocol (http) connection. In another possible example, the third party server does not support the establishment of a communication connection to the AR media processing network element. As shown in FIG. 2C, the third party server establishes a communication connection to the application server, such as a hypertext transfer protocol http connection.

[0095] As another example, one of two terminal devices in the AR communication does not have the AR capability. For example, as shown in Fig. 3, the first terminal device has the AR capability and the second terminal device does not have the AR capability. In Fig. 3, the IMS AGW1 serving the first terminal device has the AR capability.

[0096] As yet another example, one of two terminal devices in AR communication does not have AR capability. For example, as shown in Fig. 4, a first terminal device has AR capability and a second terminal device does not have AR capability. In Fig. 4, an IMS AGW2 serving a second terminal device has AR capability and an IMS AGW1 serving a first terminal device does not have AR capability.

[0097] In another possible implementation, the AR media processing network element may be a media resource function (MRF). As an example, as shown in FIG. 5, an example in which the AR media processing network element is an MRF is used, and the AR communication system includes an MRF, an IMS core, an application server (AS), and a third party server. The MRF is usually classified into a media resource function controller (MRFC) and a media resource function processor (MRFP). In a conference scenario, a terminal device communicates with another terminal device through the MRF. The MRF is a central node between session messages and media streams. In FIG. 5, a first terminal device, a second terminal device, and a third terminal device are used as an example. All of the multiple terminal devices in an AR conference may have AR capabilities, or one or more of the terminal devices may have AR capabilities, and the other terminal devices do not have AR capabilities. For example, in FIG. 5, both the first terminal device and the second terminal device have AR capabilities, and the third terminal device does not have AR capabilities. The MRFs serving the multiple terminal devices respectively may be the same or different. In FIG. 5, the MRF serving the first terminal device and the third terminal device is the same MRF, i.e., MRF 1, and the MRF serving the second terminal device is MRF 2. The different MRFs serving the multiple terminal devices performing the AR conference may all have AR processing capability, or only one or more MRFs have AR processing capability, and the other MRFs do not. Both MRF 1 and MRF 2 in FIG. 5 have AR processing capability. MRF 1 serves the first terminal device and the third terminal device, and MRF 2 serves the second terminal device.

[0098] Optionally, in this embodiment of the application, the terminal device can establish different auxiliary data channels based on the type of transmitted AR auxiliary data. For example, if pause data needs to be transmitted, auxiliary data channel 1 is established, and if operation data needs to be transmitted, auxiliary data channel 2 is established. The terminal device may alternatively transmit different types of AR auxiliary data through one auxiliary data channel.

[0099] In the following, an example in which a first terminal device and a second terminal device perform AR communication is used to describe the procedure of establishing an auxiliary data channel between the first terminal device and the AR media processing network element after the first terminal device and the second terminal device trigger AR communication. For details, please refer to Figure 6.

[0100] 601: A first terminal device sends a first request message to an IMS core network element, where the first request message is for requesting establishment of an auxiliary data channel.

[0101] The first request message may carry a first description parameter of the first terminal device, the first description parameter being used by the first terminal device to establish an auxiliary data channel with the AR media processing network element. Optionally, the first request message further carries an address of the first terminal device.

[0102] For example, the first description parameter may use a session description protocol (SDP). Of course, the first description parameter may alternatively use another description protocol. This is not specifically limited in this application. In the following description, an example in which the first description parameter uses SDP is used. For ease of distinction, the SDP information carrying the first description parameter is referred to as the first SDP information (may be referred to as the first SDP for short). In other words, the first request message carries the first SDP information of the first terminal device. The first SDP information includes description parameters for establishing an auxiliary data channel between the first terminal device and the AR media processing network element. For example, the first SDP information may include one or more of parameters such as a port number, a media stream type, and a supported codec format used by the first terminal device to transmit AR auxiliary data. The media stream type may include video (video stream), audio (audio stream), and datachannel (AR auxiliary data). For example, in this case, the first SDP information may include an m-line describing the AR assistance data.

[0103] In one example, the m-line is described as follows:

[0104] m = application 10001 user datagram protocol (UDP) / datagram transport layer security (DTLS) protocol / (Stream Control Transmission Protocol (SCTP) webrtc-datachannel).

[0105] The m line represents a description of a particular AR auxiliary data stream. 10001 represents a 5-tuple port number, UDP / DTLS / SCTP represents the transmission protocol that the AR auxiliary data complies with, and webrtc-datachannel represents the established auxiliary data channel. The first SDP information may further include lines for describing other information, such as the AR auxiliary data. In one example, the a line is written as follows:

[0106] a=dcmap:0 subprotocol="";ordered=true;max-retr=2;and label="AR-specific data", where: dcmap represents a set of parameters that need to be negotiated at both ends of the auxiliary data channel, such as whether the data channel supports reliable transmission, whether the maximum number of retransmissions is set, and the subprotocol format transferred in the data channel. As shown in the above example, 0 represents the auxiliary data channel with ID 0, subprotocol represents the protocol type of the AR auxiliary data transmitted through the auxiliary data channel with ID 0, e.g., subprotocol="" indicates that the protocol type has not been formulated at the two ends. For example, subprotocol="http" indicates that the AR auxiliary data transmitted in the auxiliary data channel is in http format. Ordered indicates whether reliable ordered transmission is performed, max-retr represents the maximum number of retransmissions, e.g., max-retr=2 indicates that the maximum number of retransmissions is 2, and label represents a description of the AR auxiliary data transmitted through the auxiliary data channel.

[0107] In another example, the first SDP information may further include:

[0108] a=dcsa:0 accept-type:text / plain, where: dcsa represents the SDP attribute of the auxiliary data channel, 0 represents the ID of a specific auxiliary data channel, and accept-type represents the format of the AR auxiliary data transmitted through the auxiliary data channel. accept-types:text / plain indicates that the AR auxiliary data is in text format.

[0109] In one example, a first terminal device may send a first request message to an IMS core network element in a procedure for establishing AR communication with a second terminal device. For example, the first request message may be a Session Initiation Protocol (SIP) invite message. The first request message may further carry address information of the first terminal device. For example, the address information of the first terminal device may include an IP address and / or a port number of the first terminal device.

[0110] Optionally, the first request message is further for requesting the establishment of a media stream channel, and the media stream channel is for transmitting a media stream for performing the AR communication between the first terminal device and the second terminal device. The first request message further carries a description parameter for establishing a media stream channel between the first terminal device and the AR media processing network element. For example, the first SDP information further includes a description parameter for establishing a media stream channel between the first terminal device and the AR media processing network element.

[0111] In another example, the first terminal device may send a first request message to an IMS core network element in a procedure to re-establish an AR session with the second terminal device, for example, the first request message may be a SIP Re-invite message.

[0112] In yet another example, the first terminal device may send a first request message to an IMS core network element in a procedure for updating an AR session with a second terminal device, for example, the first request message may be a SIP update invite message.

[0113] 602: After receiving the first request message, the IMS core network element sends a second request message to the AR media processing network element, where the second request message is for requesting the establishment of an auxiliary data channel between the AR media processing network element and the first terminal device. For example, the second request message further carries the first SDP information.

[0114] In this embodiment, the communication between the first terminal device and the second terminal device may be a point-to-point communication between the first terminal device and the second terminal device, or may be a conference communication established between the first terminal device and the second terminal device. The AR media processing network element is an IMS AGW having an AR processing capability or an MRF. For example, when the point-to-point communication is performed between the first terminal device and the second terminal device, the AR media processing network element may be an IMS AGW. When the AR conference communication is performed between the first terminal device and the second terminal device, the AR media processing network element may be an MRF. The AR media processing network element may be located in an IMS network to which the first terminal device belongs (or the first terminal device is located) and configured to provide a service to the first terminal device, or the AR media processing network element may be located in an IMS network to which the second terminal device belongs (or the second terminal device is located) and configured to provide a service to the second terminal device.

[0115] For example, when the AR media processing network element is an IMS AGW, the second request message may be an H248 protocol message, such as an H248 add request message. As another example, when the AR media processing network element is an MRF, the second request message may be a SIP invite message.

[0116] 603: Upon receiving the second request message, the AR media processing network element sends a second response message to the IMS core network element, where the second response message is a response message corresponding to the second request message. For example, the second response message indicates to the IMS core network element that the AR media processing network element agrees to establish an auxiliary data channel. For example, the second response message carries second description parameters of the AR media processing network element for establishing an auxiliary data channel. The AR media processing network element can determine second description parameters supported by the AR media processing network element from the first description parameters. The second description parameters may use the SDP protocol or other protocols. For ease of distinction, the SDP information of the AR media processing network element is referred to as second SDP information (may be referred to as second SDP for short). The second SDP information includes description parameters of the AR media processing network element for establishing an auxiliary data channel with the first terminal device. For example, the second SDP information includes parameters such as a port, a media stream type, and a supported codec format of the auxiliary data channel corresponding to the AR media processing network element. The media stream type may include video (video stream), audio (audio stream), and datachannel (AR auxiliary data). For example, in this case, the second SDP information may include an m-line for describing the AR auxiliary data obtained by the AR media processing network element through negotiation, and the media stream type of the m-line is AR auxiliary data.

[0117] 604: Upon receiving the second response message, the IMS core network element sends a first response message to the first terminal device, where the first response message carries the address of the AR media processing network element. The first response message carries second SDP information of the AR media processing network element.

[0118] In some embodiments, after receiving the first request message, the IMS core network element forwards the first request message to the application server. When determining that the first terminal device subscribes to the AR service, the application server can send address information of the first terminal device to the AR media processing network element via the IMS core network element. The application server can add the address information of the application server to the first request message and send the first request message to the IMS core network element. In this way, the IMS core network element includes the address information of the application server in the second request message sent to the AR media processing network element, so that when the AR media processing network element subsequently needs to obtain content provided by the third party server, the AR media processing network element can obtain the required content from the third party server via the AS.

[0119] For example, the application server may send a subscription query request to the HSS to query whether the subscription information of the first terminal device includes information about the subscribed AR service. The HSS may use the subscription query response to feed back the subscription information of the first terminal device to the application server. The subscription query request may be a diameter request message. The subscription query response may be a diameter response message.

[0120] In some other embodiments, after receiving the first request message, the MS core network element forwards the first request message to the application server. When determining that the first terminal device subscribes to the AR service, the application server can send the stored address information of the third party server to the AR media processing network element via the IMS core network element. The application server can add the address information of the third party server to the first request message and send the first request message to the IMS core network element. In this way, the IMS core network element includes the address information of the third party server in the second request message sent to the AR media processing network element, so that when the content provided by the third party server needs to be obtained later, the AR media processing network element can obtain the required content from the third party server.

[0121] In a possible implementation, the terminal device can know in advance whether the AR media processing network element has AR processing capability. For example, in a procedure of registering with the network side, the terminal device knows whether an AR media processing network element having AR processing capability exists in the IMS network where the terminal device is located.

[0122] In another possible implementation, before sending the second request message to the AR media processing network element, the IMS core network element first inquires whether the AR media processing network element has AR processing capability, for example, sends an inquiry message to the AR media processing network element. The inquiry message is for inquiring whether the AR media processing network element has AR processing capability. After receiving the inquiry message, the AR media processing network element sends indication information to the IMS core network element to indicate whether the AR media processing network element has AR processing capability. When it is determined that the AR media processing network element has AR processing capability, the second request message is sent to the AR media processing network element having AR processing capability.

[0123] The AR media processing network element may be an IMS AGW or an MRF having an AR processing capability. For example, the AR media processing network element is an IMS AGW. See Figures 7A and 7B. For ease of distinction, the IMS AGW serving the first terminal device is called IMS AGW1, and the IMS AGW serving the second terminal device is called IMS AGW2.

[0124] See 701a:601. Details will not be repeated here.

[0125] 702a: Upon receiving the first request message of the first terminal device, the IMS core network element sends an inquiry message 1 to the IMS AGW1, where the inquiry message 1 is to inquire whether the IMS AGW1 has AR processing capability.

[0126] For example, the inquiry message 1 may include a parameter for inquiring whether the IMS AGW 1 has AR processing capability. For example, the parameter may be “ifSupportforAR_flag”.

[0127] The inquiry message 1 can be an H248 add request. The parameter “ifSupportforAR_flag” is added to the H248 add request.

[0128] 703a: If it is determined that the IMS AGW1 has the AR processing capability, the IMS AGW1 sends a first indication to an IMS core network element, where the first indication indicates that the IMS AGW1 has the AR processing capability. For example, the first indication may be “ifSupportforAR_flag=true”. Optionally, the first indication may be carried in an H248 add response. The H248 add response includes “ifSupportforAR_flag=true”.

[0129] 704a: The IMS core network element sends a second request message to the IMS AGW1, see 602. Details are not repeated here.

[0130] For 705a and 706a, see 603 and 604. Details will not be repeated here.

[0131] For 701b in Figure 7B, see 601. Details will not be repeated here.

[0132] 702b: Upon receiving the first request message of the first terminal device, the IMS core network element sends an inquiry message 2 to the IMS AGW1, where the inquiry message 2 is to inquire whether the IMS AGW1 has AR processing capability.

[0133] For example, the inquiry message 2 may include a parameter for inquiring whether the IMS AGW1 has AR processing capability. For example, the parameter may be “ifSupportforAR_flag”. The inquiry message 2 may be an H248 add request. The parameter “ifSupportforAR_flag” is added to the H248 add request.

[0134] 703b: If the IMS AGW1 determines that it does not have AR processing capability, the IMS AGW1 sends a second indication to the IMS core network element, where the second indication indicates that the IMS AGW1 does not have AR processing capability. For example, the second indication may be “ifSupportforAR_flag=false”. Optionally, the second indication may be transmitted in a H248 add response. The H248 add response includes the parameter “ifSupportforAR_flag=false”. Alternatively, the H248 add response does not include the parameter “ifSupportforAR_flag=false” indicating that the IMS AGW1 does not have AR processing capability.

[0135] 704b: The IMS core network element sends a second request message to the IMS AGW2.

[0136] For 705b and 706b, see 603 and 604. Details will not be repeated here.

[0137] In another possible implementation, the IMS core network element includes a parameter in the second request message sent to the AR media processing network element to inquire whether the AR media processing network element has AR processing capability. For example, the second request message carries "ifSupportforAR_flag". When the AR media processing network element includes "ifSupportforAR_flag=true" in the second response message, the AR media processing network element is determined to have AR processing capability. Alternatively, when the AR media processing network element does not include the parameter "ifSupportforAR_flag=true" in the second response message but includes the second description parameter, the AR media processing network element is determined to have AR processing capability. Alternatively, when the AR media processing network element includes the parameter "ifSupportforAR_flag=false" in the second response message, the AR media processing network element is determined to not have AR processing capability.

[0138] The following describes the procedure for establishing an auxiliary data channel provided in one embodiment of this application with reference to a specific application scenario.

[0139] In a first possible application scenario, the auxiliary data channel is established in the process of creating an AR session. In this scenario, for example, the AR media processing network element having AR processing capability is the IMS AGW1 to which the first terminal device belongs.

[0140] 8A-1 and 8A-2 are schematic flowcharts of a communication method in a first possible application scenario according to this application.

[0141] 801: A first terminal device sends a first request message to an IMS core network element (eg, intermediate IMS1).

[0142] In this embodiment, for example, an IMS core network element configured to provide a service to a first terminal device is called intermediate IMS1. In Fig. 8A-1 and Fig. 8A-2, for example, the first request message is a SIP INVITE message. For ease of distinction, the first request message is called SIP INVITE1 as an example. SIP INVITE1 is for requesting the establishment of an AR session. SIP INVITE1 includes a first SDP of the first terminal device. The first SDP includes description parameters used by the first terminal device to establish an auxiliary data channel. The first SDP further includes description parameters used by the first terminal device to establish a media stream channel. SIP INVITE1 further includes address information of the first terminal device, including an IP address and a port number of the first terminal device. In Fig. 8A-1 and Fig. 8A-2, the IP address is IP1o and the port number is P1o. For example, the first SDP includes parameters such as address information (e.g., port number), type of media stream, supported codec format, etc. The media stream type may include video (video stream), audio (audio stream), and datachannel (AR auxiliary data). For example, in this case, the first SDP information may include an m-line for describing the AR auxiliary data, and may further include an m-line for describing the audio data, and an m-line for describing the video data. The m-line describes the media stream (video stream, audio stream, or stream including AR auxiliary data).

[0143] As an example, three m-lines are defined: an m-line for a video stream, an m-line for an audio stream, and an m-line for a stream containing AR auxiliary data.

[0144] m=video 49154 RTP / AVPF 101 102. The first parameter of the m line represents the media name, indicating that the type of media stream supported is video. The second parameter represents the port number, indicating that the video stream is sent through port 49154. The third parameter represents the transport protocol used, which is real-time transport protocol (RTP) / audio video profile feedback (AVPF). The fourth and fifth parameters represent the supported payload type numbers.

[0145] m=audio 49152 RTP / AVP 98 97 99. The first parameter of the m line represents the media name, indicating that the type of media stream supported is audio. The second parameter represents the port number, indicating that the audio stream is sent through port 49152. The third parameter represents the transport protocol that is used, real-time transport protocol (RTP) / audio video profile (AVP). The fourth through sixth parameters represent the supported payload type numbers.

[0146] m=application 10001 UDP / DTLS / SCTP webrtc-datachannel.

[0147] The m line represents a description of a particular AR auxiliary data stream: 10001 represents the 5-tuple port number, UDP / DTLS / SCTP represents the transport protocol that the AR auxiliary data conforms to, and webrtc-datachannel represents the auxiliary data channel that has been established.

[0148] The first SDP information may further include other information. For a specific description of the first SDP, please refer to the description of the embodiment corresponding to Figure 6. Details will not be described repeatedly in this specification.

[0149] 802: Intermediate IMS1 receives SIP INVITE1 from the first terminal device and forwards the SIP INVITE1 to the AS.

[0150] For example, after receiving SIP INVITE1, Intermediate IMS1 determines based on the subscription data of the first terminal device that the first terminal device has subscribed to the AR media extension service, and further forwards SIP INVITE1 to the AS. The AS provides a control service for the AR media extension service.

[0151] 803: The AS sends SIP INVITE2 to Intermediate IMS1.

[0152] After receiving SIP INVITE1, the AS determines that the AR service subscribed by the first terminal device triggers the corresponding AR service. The AS triggers the establishment of an AR session through Intermediate IMS1, in other words, triggers SIP INVITE2 to Intermediate IMS1. Optionally, the SIP INVITE2 triggered by the AS and the SIP INVITE1 sent by the first terminal device may be the same message or different messages.

[0153] In some embodiments, the AS identifies that the first terminal device triggers an AR service (or initiates an AR session), and the AS can record a session identifier of the AR session as an AR session and establish a binding relationship between the session identifier and the identifier of the first terminal device. After the binding relationship is established, the binding relationship can be subsequently used when the target object is obtained from the third-party server through the AS. Details will be described later and will not be repeated in this specification.

[0154] 804: Intermediate IMS1 sends an inquiry message to IMS AGW1 to inquire whether IMS AGW1 has AR capability. For ease of distinction, the inquiry message to inquire whether IMS AGW1 has AR capability is called inquiry message 1. The inquiry message 1 may be H248 add request 1 (req1). The H248 add request 1 may be for requesting establishment of a bearer. For example, the H248 add request 1 carries a parameter to inquire whether IMS AGW1 has AR processing capability. For example, the parameter to inquire whether IMS AGW1 has AR processing capability may be “IfSupportforAR_flag”.

[0155] Optionally, an H248 add request1 is received, and an outgoing termination T2 is created. Specifically, an outgoing termination of the IMS AGW1 is created. The IMS AGW1 feeds back address information corresponding to the termination T2 and indication information indicating whether the IMS AGW1 has AR processing capability to the Intermediate IMS1.

[0156] 805: After receiving the H248 add request1, the IMS AGW1 sends an inquiry response 1 to the Intermediate IMS1, where the inquiry response 1 includes indication information indicating whether the IMS AGW1 has AR processing capability. For example, the inquiry response 1 may be the H248 add response 1. The Intermediate IMS1 sends the H248 add response(resp)1, where the H248 add response 1 includes indication information and address information corresponding to the termination T2. ​​For example, the address information corresponding to the termination T2 includes an IP address and a port number. For example, the IP address is IP2o, and the port number is P2o. For example, the IMS AGW1 has AR processing capability, and the H248 add response includes “IfSupportforAR_flag=True”, or the IMS AGW1 has AR processing capability, and the H248 add response carries “IfSupportforAR_flag=False”, or does not carry a parameter. In this embodiment, for example, the IMS AGW1 has AR processing capability.

[0157] 806: After receiving the H248 add response 1, the Intermediate IMS 1 identifies that the IMS AGW 1 has AR processing capability and triggers the creation of an AR session to the Intermediate IMS 1. The Intermediate IMS 1 sends a SIP INVITE 3, which carries address information corresponding to the termination T2 of the IMS AGW 1.

[0158] It should be noted that if the Intermediate IMS 1 identifies that the IMS AGW 1 has AR processing capability, only an auxiliary data channel needs to be established between the first terminal device and the IMS AGW 1. In this case, the AR session establishment request sent to the network side of the second terminal device does not need to carry an SDP for requesting the establishment of an auxiliary data channel.

[0159] 807: The Intermediate IMS 2 sends a SIP INVITE 3 carrying address information corresponding to the termination T2 of the IMS AGW 1 to the AS 2.

[0160] 808: The AS2 sends a SIP INVITE3 carrying address information corresponding to the termination T2 of the IMS AGW1 to the Intermediate IMS2.

[0161] 809: Intermediate IMS2 sends H248 add request 2 to IMS AGW2, indicating IMS AGW2 to create originating termination T2, including, for example, IP address and port number. For example, the IP address is IP3o and the port number is P3o.

[0162] 810: The IMS AGW2 creates an originating termination T3, and sends an H248 add resp2 to the Intermediate IMS2, carrying address information corresponding to the termination T2 of the IMS AGW2.

[0163] 811: The IMS AGW2 sends a SIP INVITE 4 to a second terminal device, where the SIP INVITE 4 includes address information corresponding to the termination T2 of the IMS AGW2.

[0164] 812: The second terminal device sends a SIP 180 to the IMS AGW2, where the SIP 180 carries the address information (IP1a, P1a) of the second terminal device.

[0165] 813: Intermediate IMS2 sends H248 mod request 1 to IMS AGW2, indicating IMS AGW2 to configure originating termination T1, for example, an association relationship between the address information of T1 of IMS AGW2 and the address information of a second terminal device is established.

[0166] 814: The IMS AGW2 configures an originating termination T3, and sends a H248 mod resp 1 to the Intermediate IMS2. ​​The H248 mod resp 1 may carry address information corresponding to the termination T2 of the IMS AGW2 and address information of a second terminal device.

[0167] 815: Intermediate IMS 2 sends H248 add request 3 to IMS AGW 2, instructing IMS AGW 2 to create a terminating termination T2. ​​For example, H248 add request 3 includes address information corresponding to termination T2 of IMS AGW 1.

[0168] 816: The IMS AGW2 creates a terminating termination T2, and sends an H248 add response 2 carrying address information (IP4o, P4o) corresponding to the termination T2 of the IMS AGW2 to the Intermediate IMS2.

[0169] 817: Intermediate IMS1 receives SIP 180 from the second terminal device, for example, SIP 180 may include address information of the second terminal device. For example, the IP address of the second terminal device is IP1a, and the port number of the second terminal device is P1a. Intermediate IMS1 records the IP address and port number of the second terminal device.

[0170] 818: Intermediate IMS1 sends a configuration change request to IMS AGW1. For example, the configuration change request may be H248 mod req3. H248 mod req3 includes the IP address (IP1a) and port number (P1a) of the second terminal device. H248 mod req3 is to request IMS AGW1 to configure originating termination T2.

[0171] 819: The IMS AGW1 configures an outgoing termination T2 based on the address information of the second terminal device, and sends a configuration change response to the Intermediate IMS1, where the configuration change response may be an H248 mod response 3.

[0172] 820: Intermediate IMS1 receives the configuration change response from IMS AGW1 and sends a second request message to IMS AGW1. The second request message can be for requesting establishment of an auxiliary data channel, or for requesting to create a terminating termination T1 using, for example, H248 add req 4. The H248 add req 4 is further for requesting establishment of an auxiliary data channel. The H248 add req 4 includes the first SDP of the first terminal device and the address information of the first terminal device.

[0173] After receiving the H248 add req 4, the IMS AGW1 creates an incoming termination T1 and agrees to establish an auxiliary data channel.

[0174] 821: The IMS AGW1 sends a second response message to the Intermediate IMS1, including a response to the information about the terminating termination T1, and may further include a response to the establishment of the auxiliary data channel. The second response message may be H248 add response 4. The H248 add response 4 includes the second SDP information of the IMS AGW1 and the address information of the IMS AGW1. For example, the address information of the IMS AGW1 includes the IP address (IP2a) and the port number (P2a) of the IMS AGW1. Optionally, the H248 add response 4 may further include the address information of the first terminal device.

[0175] For example, the second SDP information includes an m-line to describe an auxiliary data channel that needs to be established by the IMS AGW1. In one example, the response of the IMS AGW1 to the m-line of the first terminal device can be described as follows:

[0176] m=application 10001 UDP / DTLS / SCTP webrtc-datachannel.

[0177] 822: Intermediate IMS1 records address information of the terminating T1 of IMS AGW1 and determines second SDP information for establishing an auxiliary data channel, and a first response message sent to the AS includes the second SDP information and the address information of IMS AGW1. For example, the first response message may be SIP 180. The SIP 180 includes the second SDP information and the address information of IMS AGW1.

[0178] 823: AS forwards SIP 180 to Intermediate IMS1.

[0179] 824: Intermediate IMS1 forwards the SIP 180 to the first terminal device.

[0180] In some embodiments, in step 803, after receiving SIP INVITE1, when determining that the first terminal device subscribes to the AR service, the AS may send address information of the AS to IMS AGW1 via Intermediate IMS1. The AS may add the address information of the AS to SIP INVITE1 and send SIP INVITE1 to Intermediate IMS1. In this way, the second request message (H248 add request) sent by Intermediate IMS1 to IMS AGW1 carries the address information of the AS, so that when IMS AGW1 needs to continue to obtain content provided by the third party server, IMS AGW1 can obtain the required content from the third party server through the AS.

[0181] For example, the AS may send a subscription query request to the HSS to query whether the subscription information of the first terminal device includes information about the subscribed AR service. The HSS may feed back the subscription information of the first terminal device to the AS using a subscription query response. The subscription query request may be a diameter request message. The subscription query response may be a diameter response message.

[0182] In some other embodiments, after receiving the SIP INVITE1, the Intermediate IMS1 forwards the SIP INVITE1 to the AS. When determining that the first terminal device has subscribed to the AR service, the AS can send the stored address information of the third party server to the IMS AGW1 via the Intermediate IMS1. The AS can add the address information of the third party server to a first request message and send the first request message to the Intermediate IMS1. In this way, the second request message (H248 add request) sent by the Intermediate IMS1 to the IMS AGW1 carries the address information of the third party server, so that when the IMS AGW1 subsequently needs to obtain content provided by the third party server, the IMS AGW1 can obtain the required content from the third party server.

[0183] The above-mentioned terminations T1 and T2 will be described with reference to Fig. 8B. The termination T1 is on the IMS AGW and is a termination for connecting to a terminal device through an access network. The termination T2 is on the IMS AGW and is a termination for connecting to an IMS core network. The IMS AGW can establish a connection with a communication peer terminal device through the IMS core network.

[0184] In a second possible application scenario, the establishment of the auxiliary data channel is triggered by triggering a process of re-establishing an AR session. In this scenario, an example is used in which the IMS AGW1 to which the first terminal device belongs has AR processing capability.

[0185] FIG. 9 is a schematic flowchart of a communication method in a second possible application scenario according to this application.

[0186] 901: After an AR session is established between a first terminal device and a second terminal device, the first terminal device sends a first request message to an IMS core network element (e.g., intermediate IMS1). In FIG. 9, an example is used in which the first request message is a SIP re-establishment request (Re-INVITE1). The SIP Re-INVITE1 is for requesting the creation of an auxiliary data channel. The SIP Re-INVITE1 includes a first SDP of the first terminal device. For a description of the first SDP of the first terminal device, please refer to the description of the embodiment corresponding to FIG. 6. Details will not be repeated in this specification.

[0187] 902: Intermediate IMS1 receives SIP Re-INVITE1 from the first terminal device and forwards the SIP Re-INVITE1 to the AS.

[0188] 903: The AS sends a SIP Re-INVITE1 to the Intermediate IMS1.

[0189] 904: Intermediate IMS1 sends an inquiry message 1 to IMS AGW1. The inquiry message 1 may be carried in an H248 mod req. The H248 mod req is for requesting the establishment of a bearer. For example, the H248 mod req may further carry a parameter for inquiring whether the IMS AGW1 has AR processing capability. For example, the parameter for inquiring whether the IMS AGW1 has AR processing capability may be “IfSupportforAR_flag”.

[0190] Optionally, after receiving the H248 mod req, the IMS AGW1 indicates to the Intermediate IMS1 whether the IMS AGW1 has AR processing capability.

[0191] 905: After receiving the H248 mod request, the IMS AGW1 sends an inquiry response 1 to the Intermediate IMS1, where the inquiry response 1 includes indication information indicating whether the IMS AGW1 has AR processing capability. For example, the Intermediate IMS1 sends an H248 mod response, where the H248 mod response includes indication information. For example, the IMS AGW1 has AR processing capability, and the H248 mod response includes “IfSupportforAR_flag=True”, or the IMS AGW1 has AR processing capability, and the H248 mod response carries “IfSupportforAR_flag=False”, or does not carry any parameter. In this embodiment, for example, the IMS AGW1 has AR processing capability.

[0192] 906: After receiving the H248 mod response, the Intermediate IMS1 identifies that the IMS AGW1 has AR processing capability, in other words, only an auxiliary data channel needs to be established between the first terminal device and the IMS AGW1. In this case, the AR session establishment request sent to the second terminal device does not need to carry a request to establish an auxiliary data channel. The AR session establishment request may be a SIP Re-INVITE2.

[0193] For example, Intermediate IMS1 sends SIP Re-INVITE2 to the second terminal device via Intermediate IMS2 to which the second terminal device belongs.

[0194] 907: Intermediate IMS1 receives a SIP response, for example SIP 200 OK, from the second terminal device.

[0195] 908: Intermediate IMS1 sends a second request message to IMS AGW1, where the second request message may be for requesting establishment of an auxiliary data channel, and further for modifying the terminating termination T1 information. For example, Intermediate IMS1 sends a request to modify the terminating termination T1 information to IMS AGW1 using H248 add req. The H248 add req is further for requesting establishment of an auxiliary data channel. The H248 add req includes a first SDP of the first terminal device.

[0196] After receiving the H248 add req, the IMS AGW1 modifies the incoming termination T1 information and agrees to establish an auxiliary data channel.

[0197] 909: The IMS AGW1 sends a response to the Intermediate IMS1 to modify the terminating T1 information. The response to change the terminating T1 information may be an H248 add response. The H248 add response includes the second SDP information of the IMS AGW1. For a description of the second SDP, please refer to the description of the embodiment corresponding to FIG. 6. Details will not be repeated in this specification.

[0198] 910: Intermediate IMS1 extracts the second SDP information of IMS AGW1 and sends an AR session establishment response including the second SDP information to the AS. For example, the AR session establishment response may be a SIP 200 OK. The SIP 200 OK includes the second SDP information.

[0199] 911: The AS forwards the SIP 200 OK to Intermediate IMS1.

[0200] 912: Intermediate IMS1 forwards the SIP 200 OK to the first terminal device.

[0201] In some embodiments, in step 903, after receiving the SIP Re-INVITE1, when determining that the first terminal device subscribes to the AR service, the AS may send address information of the AS to the IMS AGW1 via the Intermediate IMS1. The AS may add the address information of the AS to the SIP Re-INVITE1 and send the SIP Re-INVITE1 to the Intermediate IMS1. In this way, the second request message (H248 add request) sent by the Intermediate IMS1 to the IMS AGW1 carries the address information of the AS, so that when the IMS AGW1 needs to continue to obtain content provided by the third party server, the IMS AGW1 can obtain the required content from the third party server through the AS.

[0202] In some other embodiments, after receiving the SIP Re-INVITE1, the Intermediate IMS1 forwards the SIP Re-INVITE1 to the AS. When determining that the first terminal device has subscribed to the AR service, the AS can send the stored address information of the third party server to the IMS AGW1 via the Intermediate IMS1. The AS can add the address information of the third party server to a first request message and send the first request message to the Intermediate IMS1. In this way, the second request message (H248 add request) sent by the Intermediate IMS1 to the IMS AGW1 carries the address information of the third party server, so that when the IMS AGW1 subsequently needs to obtain content provided by the third party server, the IMS AGW1 can obtain the required content from the third party server.

[0203] In a third possible application scenario, the auxiliary data channel is established in the process of creating an AR session, for example, an IMS AGW1 to which a first terminal device belongs does not have AR processing capability, and an IMS AGW2 to which a second terminal device belongs has AR processing capability.

[0204] 10A and 10B are schematic flowcharts of a communication method in a third possible application scenario according to this application.

[0205] 1001 to 1004: see 801 to 804. Details will not be repeated here.

[0206] 1005: After receiving the H248 add request, the IMS AGW1 sends to the Intermediate IMS1 indication information indicating whether the IMS AGW1 has AR processing capability. For example, the Intermediate IMS1 sends a H248 add response, and the H248 add response carries indication information and address information corresponding to the termination T2. ​​For example, the address information corresponding to the termination T2 includes an IP address and a port number. For example, the IP address is IP2o, and the port number is P2o. For example, the IMS AGW1 has AR processing capability, and the H248 add response includes "IfSupportforAR_flag=True", or the IMS AGW1 has AR processing capability, and the H248 add response carries "IfSupportforAR_flag=False", or does not carry this parameter. In this embodiment, for example, the IMS AGW1 does not have AR processing capability.

[0207] After receiving the H248 add response, the Intermediate IMS1 identifies that the IMS AGW1 does not have AR processing capability, and sends a second request message (SIP INVITE2) to the IMS AGW2 via the Intermediate IMS2 and the AS to which the second terminal device belongs. The SIP INVITE2 sent to the second terminal device does not need to carry a request to request the establishment of an auxiliary data channel. The SIP INVITE2 sent to the second terminal device carries address information corresponding to the termination T2 of the IMS AGW1.

[0208] 1006-1008: 806-808. Details will not be repeated here.

[0209] 1009: After receiving the SIP INVITE 3 from the AS, the Intermediate IMS 2 may send an inquiry message 2 to the IMS AGW 2.

[0210] The inquiry message 2 may be carried in H248 add req 2. For example, the H248 add req 2 may further carry a parameter for inquiring whether the IMS AGW2 has AR processing capability. For example, the parameter for inquiring whether the IMS AGW2 has AR processing capability may be “IfSupportforAR_flag”. The H248 add req 2 further indicates to create an output termination T2. ​​For details, see 809. Details are not repeated in this specification.

[0211] Optionally, after receiving the H248 add req 2, the IMS AGW2 indicates to the Intermediate IMS2 whether the IMS AGW2 has AR processing capability.

[0212] 1010: After receiving the H248 add request 2, the IMS AGW2 sends an inquiry response 2 to the Intermediate IMS2, where the inquiry response 2 includes the indication information 2 indicating whether the IMS AGW2 has the AR processing capability. For example, the IMS AGW2 sends the H248 add response 2 (i.e., the inquiry response 2), where the H248 add response 2 includes the indication information 2. For example, the IMS AGW2 has the AR processing capability, and the H248 add response 2 includes "IfSupportforAR_flag=True", or the IMS AGW2 has the AR processing capability, and the H248 add response carries "IfSupportforAR_flag=False", or does not carry this parameter. In this embodiment, for example, the IMS AGW2 has the AR processing capability. The H248 add response 2 further includes the address information of T2.

[0213] 1011-1014: 811-814. Details will not be repeated here.

[0214] 1015: Intermediate IMS2 sends H248 add request 3 to IMS AGW2, indicating IMS AGW2 to create a terminating termination T1. The H248 add request 3 is further for requesting establishment of an auxiliary data channel with a first terminal device. The H248 add request 3 includes a first SDP of the first terminal device. For example, the H248 add request 3 further includes address information corresponding to a termination T2 of the IMS AGW1.

[0215] 1016: The IMS AGW2 creates a terminating termination T2 and sends an H248 add resp 3 to the Intermediate IMS2, where the H248 add resp 2 includes an acknowledgement for creating an auxiliary data channel, such as a second SDP of the IMS AGW2, and further includes address information (IP4o, P4o) corresponding to the termination T2 of the IMS AGW2.

[0216] 1017: Intermediate IMS2 sends SIP 180 to Intermediate IMS1.

[0217] 1018: Intermediate IMS1 sends a configuration change request to IMS AGW1. For example, the configuration change request may be H248 mod req3, which is to request IMS AGW1 to configure originating termination T2.

[0218] 1019: The IMS AGW1 configures an outgoing termination T2, and sends a configuration change response message to the Intermediate IMS1, where the configuration change response message may be an H248Mod response3.

[0219] 1020: Intermediate IMS1 receives a configuration change response message from IMS AGW1, and then initiates creation of a terminating termination T1 to IMS AGW1. For example, the request message for creating a terminating termination T1 carries a request for establishing an auxiliary data channel and address information of a first terminal device. The request message for creating a terminating termination T1 may be H248 add req4.

[0220] 1021: The IMS AGW1 sends a H248 add response 4 to the Intermediate IMS1. The H248 add response 4 includes address information of the IMS AGW1. For example, the address information of the IMS AGW1 includes an IP address (IP2a) and a port number (P2a) of the IMS AGW1. Optionally, the H248 add response 4 may further include address information of the first terminal device.

[0221] 1022: Intermediate IMS1 records the address information of the terminating T1 of IMS AGW1, and sends SIP 180 including the second SDP information of IMS AGW2 and the address information of IMS AGW1 to the AS.

[0222] 1023: The AS forwards the SIP 180 to Intermediate IMS1.

[0223] 1024: Intermediate IMS1 forwards the SIP 180 to the first terminal device.

[0224] In some embodiments, after receiving the SIP INVITE3, the AS2 may send address information of the AS2 to the IMS AGW2 through the Intermediate IMS 2. The AS2 may add the address information of the AS2 to the SIP INVITE3 and send the SIP INVITE3 to the Intermediate IMS 2. In this way, the H 248 add req3 sent by the Intermediate IMS 2 to the IMS AGW2 carries the address information of the AS2, so that when the IMS AGW2 needs to obtain content provided by the third-party server later, the IMS AGW2 can obtain the required content from the third-party server through the AS.

[0225] In some other embodiments, after receiving the SIP INVITE3, the Intermediate IMS2 forwards the SIP INVITE3 to the AS2. The AS2 can send the stored address information of the third party server to the IMS AGW2 via the Intermediate IMS2. ​​The Intermediate IMS2 can add the address information of the third party server to the H248 add req 3 and send the H248 add req 3 to the IMS AGW2, so that when the IMS AGW2 needs to obtain content provided by the third party server thereafter, the IMS AGW1 can obtain the required content from the third party server.

[0226] In a fourth possible application scenario, the auxiliary data channel is established in the process of creating an AR conference. In this scenario, an example is used in which the MRF to which the first terminal device belongs has AR processing capability.

[0227] FIG. 11 is a schematic flowchart of a communication method in a fourth possible application scenario according to this application.

[0228] 1101 to 1103: 801 to 803. Details will not be repeated here.

[0229] 1104: The Intermediate IMS1 identifies that the first terminal device initiates the AR conference and sends a second request message to the MRF, where the second request message includes the first SDP of the first terminal device. The second request message may be a SIP INVITE2.

[0230] 1105: If it is determined that the MRF has the AR processing capability, the MRF sends a second response message to the Intermediate IMS1, and the second response message includes the second SDP of the MRF. If it is determined that the MRF does not have the AR processing capability, the MRF ignores the request of the first terminal device to establish an auxiliary data channel. For example, the second response message may be SIP 180.

[0231] 1106: Intermediate IMS1 sends a first response message to the AS, where the first response message may be SIP 180. The first response message includes the second SDP of the MRF.

[0232] 1107: The AS sends a first response message to Intermediate IMS1.

[0233] 1108: Intermediate IMS1 sends a first response message to the first terminal device.

[0234] In some embodiments, in step 1103, after receiving SIP INVITE1, when it determines that the first terminal device subscribes to the AR service, the AS may send address information of the AS to the MRF via Intermediate IMS1. The AS may add the address information of the AS to SIP INVITE1 and send SIP INVITE1 to Intermediate IMS1. In this way, the second request message (SIP INVITE2) sent by intermediate MRF 1 to the MRF carries the address information of the AS, so that when the MRF subsequently needs to obtain content provided by the third party server, the MRF can obtain the required content from the third party server via the AS.

[0235] In some other embodiments, after receiving SIP INVITE1, Intermediate IMS1 forwards SIP INVITE1 to the AS. When determining that the first terminal device subscribes to the AR service, the AS can send the stored address information of the third party server to the MRF via Intermediate IMS1. The AS can add the address information of the third party server to the first request message and send the first request message to Intermediate IMS1. In this way, the second request message (SIP INVITE2) sent by Intermediate IMS1 to the MRF carries the address information of the third party server, so that when IMS AGW1 needs to obtain content provided by the third party server thereafter, the MRF can obtain the required content from the third party server.

[0236] With reference to an embodiment, the following describes a procedure for enhancing a media stream by completing the establishment of an auxiliary data channel in an embodiment of this application.

[0237] 12A illustrates a procedure for enhancing a media stream in a process of performing AR communication between a first terminal device and a second terminal device. An auxiliary data channel is established between the first terminal device and an AR media processing network element. A media stream channel is further established between the first terminal device and the second terminal device. The media stream channel passes through the AR media processing network element.

[0238] 1201: In a process of performing AR communication with a second terminal device, a first terminal device transmits AR assistance data to an AR media processing network element through an assistance data channel. The AR media processing network element is located in an IMS network of the first terminal device or in an IMS network of the second terminal device. The AR media processing network element may be an IMS AGW or an MRF.

[0239] 1202: A first terminal device transmits a media stream to an AR media processing network element through a media stream channel. The media stream is generated in the process of performing AR communication between the first terminal device and a second terminal device.

[0240] 1203: The AR media processing network element performs media enhancement processing on the media stream based on the AR auxiliary data to obtain an AR media stream.

[0241] 1204: The AR media processing network element transmits the AR media stream to the first terminal device, which receives the AR media stream.

[0242] Optionally, at 1205, the AR media processing network element transmits the AR media stream to a second terminal device through a media stream channel. The second terminal device receives the AR media stream.

[0243] For example, the AR media processing network element is an IMS AGW. The IMS AGW serving the first terminal device is called IMS AGW1, and the IMS AGW serving the second terminal device is called IMS AGW2. As shown in FIG. 12 B, both IMS AGW1 and IMS AGW2 have AR processing capabilities. In this application, the AR auxiliary data proximity processing principle is used. If the AR auxiliary data (non-media data) received by IMS AGW1 is data related to the first terminal device, such as the pose data of the user of the first terminal device, the AR auxiliary data is only used to perform pose rendering on the media stream shared by the first terminal device. Thus, IMS AGW1 obtains 1{a+b} (AR media stream) through combining and rendering based on 1 (original media stream), a (non-media data such as the pose data of the user of the first terminal device), and b (non-media data such as the user annotation data obtained by the first terminal device), and returns the AR media stream to the first terminal device. IMS AGW1 then obtains 1{b} (AR media stream) through rendering based on 1 (original media stream) and b (non-media data such as user annotation data obtained by UE 1), and sends the AR media stream to IMS AGW2. IMS AGW2 finally obtains media stream 1{b+c} (AR media stream) through rendering based on the received media stream and c (non-media data) of UE 2, and sends the AR media stream to UE 2. As shown in FIG. 12C, if IMS AGW2 does not have AR media processing capability, IMS AGW2 forwards c (non-media data) of UE 2 to IMS AGW1 for media enhancement processing.

[0244] With reference to a particular application scenario, in the following a solution for enhancing a media stream through an auxiliary data channel is described.

[0245] For example, a user of a first terminal device shares a video with a user of a second terminal device. The user of the first terminal device and the user of the second terminal device can simultaneously watch the video shared by the first terminal device. In the following description, for example, the AR media processing network element is an IMS AGW.

[0246] Please refer to Figure 13. For example, both the IMS AGW1 to which the first terminal device belongs and the IMS AGW2 to which the second terminal device belongs have AR processing capabilities. An auxiliary data channel 1 has been established between the first terminal device and the IMS AGW1 to which the first terminal device belongs. An auxiliary data channel 2 has been established between the second terminal device and the IMS AGW2 to which the second terminal device belongs. In addition, a media stream channel is established between the first terminal device and the second terminal device. The media stream channel can be considered to be divided into three segments, including a media stream channel between the first terminal device and the IMS AGW1, a media stream channel between the IMS AGW1 and the IMS AGW2, and a media stream channel between the IMS AGW2 and the second terminal device, respectively. The first terminal device has AR capabilities. For example, the first terminal device supports a user in annotating a shared picture. For ease of distinction, the user of the first terminal device is referred to as User 1 and the user of the second terminal device is referred to as User 2.

[0247] 1301: A first terminal device sends a media stream to the IMS AGW1 through a media stream channel.

[0248] 1302: The first terminal device acquires pose data 1 generated by annotations made by a user 2 on a display interface corresponding to the media stream, and transmits the pose data 1 as AR auxiliary data to the IMS AGW1 via an auxiliary data channel 1.

[0249] 1303: The IMS AGW1 renders the media stream based on the pose data1 to obtain the AR media stream1.

[0250] 1304a: IMS AGW1 sends AR media stream 1 to IMS AGW2.

[0251] 1304b: IMS AGW1 sends AR media stream 1 to the first terminal device.

[0252] 1305: The IMS AGW2 sends the AR media stream 1 to the second terminal device through a media stream channel.

[0253] In this way, both user 1 at the first terminal device and user 2 at the second terminal device can see the display interface on which user 1 makes annotations.

[0254] In some embodiments, the second terminal device also has AR capabilities, for example to support the user in annotating the shared picture.

[0255] 1306: The second terminal device obtains pose data 2 generated by annotations made by user 2 on a display interface corresponding to the media stream, and sends the pose data 2 as AR auxiliary data to IMS AGW2 through auxiliary data channel 2.

[0256] 1307: The IMS AGW2 renders the media stream based on the pose data2 to obtain the AR media stream2.

[0257] 1308a: IMS AGW2 sends AR media stream 2 to IMS AGW1.

[0258] 1308b: IMS AGW2 sends AR media stream 2 to the second terminal device.

[0259] 1309: IMS AGW1 transmits AR media stream 2 to the first terminal device through media stream channel 1. In this way, both user 1 of the first terminal device and user 2 of the second terminal device can see the display interface on which user 2 makes annotations.

[0260] Please refer to Figure 14. For example, the IMS AGW1 to which the first terminal device belongs does not have AR processing capability, and the IMS AGW2 to which the second terminal device belongs has AR processing capability. An auxiliary data channel 1 is established between the first terminal device and the IMS AGW2 to which the second terminal device belongs. An auxiliary data channel 2 is established between the second terminal device and the IMS AGW2 to which the second terminal device belongs. In addition, a media stream channel is established between the first terminal device and the second terminal device. The media stream channel can be considered to be divided into three segments, including a media stream channel between the first terminal device and the IMS AGW1, a media stream channel between the IMS AGW1 and the IMS AGW2, and a media stream channel between the IMS AGW2 and the second terminal device, respectively. The first terminal device has AR capability. For example, the first terminal device supports a user in annotating a shared picture. For ease of distinction, the user of the first terminal device is referred to as User 1 and the user of the second terminal device is referred to as User 2.

[0261] 1401: A first terminal device sends a media stream to the IMS AGW1 through a media stream channel.

[0262] 1402: The first terminal device acquires pose data 1 generated by annotations made by a user 2 on a display interface corresponding to the media stream, and transmits the pose data 1 as AR auxiliary data to an IMS AGW2 via an auxiliary data channel 1.

[0263] 1403: IMS AGW2 renders the media stream based on the pose data1 to obtain AR media stream1.

[0264] 1404a: IMS AGW2 sends AR media stream 1 to IMS AGW1 through a media stream channel.

[0265] 1405: IMS AGW1 sends AR media stream 1 to the first terminal device through a media stream channel.

[0266] 1404b: IMS AGW2 sends AR media stream 1 to a second terminal device.

[0267] In this way, both user 1 at the first terminal device and user 2 at the second terminal device can see the display interface on which user 1 makes annotations.

[0268] In some embodiments, the second terminal device also has AR capabilities, for example to support the user in annotating the shared picture.

[0269] 1406: The second terminal device obtains pose data 2 generated by annotations made by user 2 on a display interface corresponding to the media stream, and sends the pose data 2 as AR auxiliary data to the IMS AGW2 via auxiliary data channel 2.

[0270] 1407: The IMS AGW2 renders the media stream based on the pose data2 to obtain the AR media stream2.

[0271] 1408a: IMS AGW2 sends AR media stream 2 to IMS AGW1.

[0272] 1408b: IMS AGW2 sends AR media stream 2 to a second terminal device.

[0273] 1409: IMS AGW1 transmits AR media stream 2 to the first terminal device through media stream channel 1. In this way, both user 1 of the first terminal device and user 2 of the second terminal device can see the display interface on which user 2 makes annotations.

[0274] With reference to one embodiment, the following describes the procedure of obtaining a target object in this embodiment of this application.

[0275] FIG. 15 is a schematic flowchart of obtaining possible target objects according to an embodiment of the present application.

[0276] 1501: In the process of AR communication between the first terminal device and the second terminal device, the first terminal device sends first AR assistance data to an AR media processing network element through an assistance data channel. The first AR assistance data includes information about a target object that needs to be used in the AR communication. The AR media processing network element referred to in this specification is an IMS access gateway having an AR processing capability or an MRF. It can be understood that an assistance data channel is established between the AR media processing network element and the first terminal device. The AR media processing network element can be located in the IMS network to which the first terminal device belongs, i.e., the IMS AGW or MRF corresponding to the first terminal device. The AR media processing network element can be located in the IMS network to which the second terminal device belongs, i.e., the IMS AGW or MRF corresponding to the second terminal device.

[0277] 1502: Upon receiving first AR auxiliary data from the first terminal device through the auxiliary data channel, the AR media processing network element obtains a target object from a third party server based on the first AR auxiliary data. It can be understood that the target object is used to enhance a media stream for AR communication between the first terminal device and the second terminal device.

[0278] For example, in some application scenarios, it is necessary to use virtual objects, such as 2D / 3D objects, but the terminal device usually does not have the ability to store a large amount of 2D / 3D objects. Therefore, in the above method, a request of a terminal device user can be received through an auxiliary data channel, and the 2D / 3D object can be obtained from a third-party server.

[0279] For example, in some application scenarios, when the actual background picture required by the foreground object is relatively fixed, in order to reduce the uplink bandwidth pressure on the terminal device side, the AR media processing network element can render the foreground picture and the background picture. In this case, the terminal device can send request information of the actual background picture to the AR media processing network element through an auxiliary data channel, so that the AR media processing network element obtains the actual background picture from a third-party server based on the request information, further renders the actual background picture and the foreground media stream sent by the terminal device, and sends the media stream obtained by rendering the rendered background picture and the foreground media stream to the terminal device.

[0280] In a possible implementation, the AR media processing network element acquires the address information of the AS in advance, and when acquiring the target object, the AR media processing network element acquires the target object from the third party server through the AS. For a specific acquisition method, please refer to the above description. The details will not be described again in this specification.

[0281] In another possible implementation form, the AR media processing network element acquires address information of the third-party server in advance, so that when acquiring the target object, the AR media processing network element can directly acquire the target object from the third-party server based on the address information of the third-party server.

[0282] With reference to a specific embodiment, the procedure for obtaining a virtual object is described below, and for details, please refer to Figures 16 and 17.

[0283] See Figure 16. For example, the AR media processing network element obtains a virtual object from a third party server through an AS. The AR media processing network element is an IMS AGW / MRF. The IMS AGW can be the IMS AGW to which the first terminal device or the second terminal device in the communication peer belongs. The MRF may be the MRF to which the first terminal device or the second terminal device in the communication peer belongs.

[0284] 1601: A first terminal device sends information about a virtual object, such as an identifier of the virtual object or a type of the virtual object, to the IMS AGW / MRF through an auxiliary data channel.

[0285] 1602: Upon receiving the information about the virtual object, the IMS AGW / MRF sends a request message 1 to the application server, where the request message 1 includes the information about the virtual object. The type of the virtual object is used as an example. For example, the request message 1 may be an HTTP message type. Of course, another message type may be used instead. This is not limited in this application. For example, the request message 1 may be an HTTP request (request, req) message 1. For example, the request message 1 includes an identifier of the terminal device.

[0286] 1603: When the application server receives the request message 1 and determines that the first terminal device needs to request a virtual object, it forwards the request message 1 to the third party server. For example, the application server can extract an identifier of the terminal device from the request message 1, and determine an identifier of a session established by the terminal device based on the identifier of the terminal device. The application server can establish a communication connection, for example, an HTTP connection, to the third party server based on the identifier of the session, so that the application server forwards the request message 1 to the third party server to obtain the virtual object from the third party server. The session identifier of the first terminal device can be added to the request message 1 forwarded to the third party server.

[0287] 1604: The third party server receives the request message 1, and obtains a plurality of virtual object identifiers that satisfy the type of the virtual object based on the type of the virtual object. An example is used in which the virtual object list includes a plurality of virtual object identifiers. Of course, the plurality of virtual object identifiers may alternatively be transmitted in another form. This is not specifically limited in this application. The third party server sends a response message 1 to the application server. The response message 1 may include a virtual object list. The response message 1 may be an HTTP message type. For example, the response message 1 may be an HTTP response (response, resp) message 1. For example, the response message 1 includes a session identifier.

[0288] 1605: The application server forwards the response message 1 to the IMS AGW / MRF. The application server determines an identifier of a terminal device based on the session identifier in the response message 1, and adds the identifier of the first terminal device to the response message 1 forwarded to the IMS AGW / MRF.

[0289] 1606: The IMS AGW / MRF obtains the virtual object list from the response message 1 from the application server, and sends the virtual object list to the first terminal device through the auxiliary data channel. Optionally, after obtaining the virtual object list, the IMS AGW / MRF stores the virtual object list. Thereafter, when receiving virtual object information corresponding to the virtual object list through the auxiliary data channel, the IMS AGW / MRF may no longer obtain the virtual object list from the third party server. It can be understood that in step 1602, when it is determined that the IMS AGW / MRF stores the virtual object list corresponding to the virtual object information, the IMS AGW / MRF may no longer perform the subsequent step of obtaining the virtual object list.

[0290] 1607: The first terminal device sends a first virtual object identifier to the IMS AGW / MRF through an auxiliary data channel, where the first virtual object identifier is one in the virtual object list.

[0291] 1608: Upon receiving the first virtual object identifier, the IMS AGW / MRF sends a request message 2 to the application server, where the request message 2 includes the first virtual object identifier. For example, the request message 2 may be of an HTTP message type. For example, the request message 2 may be an HTTP request (request, req) message 2.

[0292] 1609: The application server receives the request message 2 and forwards the request message 2 to the third party server.

[0293] 1610: A third party server receives the request message 2 and obtains a first virtual object corresponding to the first virtual object identifier. The third party server sends a response message 2 to the application server. The response message 2 may include the first virtual object. The response message 2 may be an HTTP message type. For example, the response message 2 may be an HTTP response (response, resp) message 2.

[0294] 1611: The application server forwards the response message 2 to the IMS AGW / MRF.

[0295] 1612: The IMS AGW / MRF obtains the first virtual object from the response message 2, and sends the first virtual object to the first terminal device through the auxiliary data channel.

[0296] See Figure 17. For example, the AR media processing network element directly obtains a virtual object from a third-party server. The AR media processing network element is an IMS AGW / MRF. The IMS AGW can be the IMS AGW to which the first terminal device or the second terminal device in the communication peer belongs. The MRF may be the MRF to which the first terminal device or the second terminal device in the communication peer belongs.

[0297] 1701: A first terminal device sends information about a virtual object, such as an identifier of the virtual object or a type of the virtual object, to the IMS AGW / MRF through an auxiliary data channel.

[0298] 1702: Upon receiving the information about the virtual object, the IMS AGW / MRF sends to the third party server a request message 11. The request message 11 may include information about the virtual object.

[0299] 1703: The third party server receives the request message 11, and obtains a plurality of virtual object identifiers that satisfy the type of the virtual object based on the type of the virtual object. An example is used in which the virtual object list includes a plurality of virtual object identifiers. Of course, the plurality of virtual object identifiers may alternatively be transmitted in another form. This is not specifically limited in this application. The third party server sends a response message 11 to the IMS AGW / MRF. The response message 11 may include a virtual object list. The response message 11 may be of an HTTP message type. For example, the response message 11 may be an HTTP response (resp) message 11.

[0300] 1704: The IMS AGW / MRF retrieves the virtual object list from the response message 11 and sends the virtual object list to the first terminal device through the auxiliary data channel. Optionally, after retrieving the virtual object list, the IMS AGW / MRF stores the virtual object list. Thereafter, when receiving virtual object information corresponding to the virtual object list through the auxiliary data channel, the IMS AGW / MRF may no longer retrieve the virtual object list from the third party server. It may be understood that in step 1702, when it is determined that the IMS AGW / MRF has stored the virtual object list corresponding to the virtual object information, the IMS AGW / MRF may no longer perform the subsequent step of retrieving the virtual object list.

[0301] 1705: The first terminal device sends a first virtual object identifier to the IMS AGW / MRF through an auxiliary data channel, where the first virtual object identifier is one in the virtual object list.

[0302] 1706: Upon receiving the first virtual object identifier, the IMS AGW / MRF sends a request message 12 to the third party server.

[0303] 1707: The third party server receives the request message 12 and obtains a first virtual object corresponding to the first virtual object identifier. The third party server sends a response message 12 to the IMS AGW / MRF. The response message 12 may include the first virtual object. The response message 12 may be of an HTTP message type. For example, the response message 12 may be an HTTP response (response, resp) message 12.

[0304] 1708: The IMS AGW / MRF obtains the first virtual object from the response message 12, and sends the first virtual object to the first terminal device through the auxiliary data channel.

[0305] With reference to a specific embodiment, the following describes a media augmentation procedure for obtaining and using real background objects, see Figures 18 and 19 for details.

[0306] See Figure 18. For example, the AR media processing network element obtains the actual background object from a third party server through an AS. The AR media processing network element is an IMS AGW / MRF. The IMS AGW can be the IMS AGW to which the first terminal device or the second terminal device in the communication peer belongs. The MRF may be the MRF to which the first terminal device or the second terminal device in the communication peer belongs.

[0307] 1801: A first terminal device sends information about an actual background object, such as an identifier of the actual background object or a geographical location of the actual background object, to an IMS AGW / MRF through an auxiliary data channel.

[0308] 1802: The first terminal device sends media stream 1 generated by the first terminal device to the IMS AGW / MRF through a media stream channel.

[0309] 1803: Upon receiving the information about the actual background object, the IMS AGW / MRF sends a request message 21 to the application server, where the request message 21 includes information about the actual background object. As an example, the geographical location of the actual background object is used. For example, the request message 21 may be of an HTTP message type. Of course, another message type may be used instead. This is not limited in this application. For example, the request message 21 may be an HTTP request (request, req) message 21.

[0310] 1804: When the application server receives the request message 21 and determines that the first terminal device needs to request an actual background object, it forwards the request message 21 to the third-party server.

[0311] 1805: The third party server receives the request message 21 and obtains actual background objects based on the geographical locations of the actual background objects. The third party server sends a response message 21 to the application server. The response message 21 may include an actual background object list. The response message 21 may be of an HTTP message type. For example, the response message 21 may be an HTTP response (response, resp) message 21.

[0312] 1806: The application server forwards the response message 21 to the IMS AGW / MRF.

[0313] 1807: The IMS AGW / MRF obtains the actual background object from the response message 21, and renders the actual background object into the media stream 1 to obtain the AR media stream 2.

[0314] 1808: The IMS AGW / MRF sends the AR media stream 2 to the first terminal device through the media stream channel.

[0315] See Figure 19. For example, the AR media processing network element directly obtains the actual background object from a third-party server. The AR media processing network element is an IMS AGW / MRF. The IMS AGW can be the IMS AGW to which the first terminal device or the second terminal device in the communication peer belongs. The MRF may also be the MRF to which the first terminal device or the second terminal device in the communication peer belongs.

[0316] 1901: A first terminal device sends information about a real background object, such as an identifier of the real background object or a geographical location of the real background object, to an IMS AGW / MRF through an auxiliary data channel.

[0317] 1902: The first terminal device sends media stream 1 generated by the first terminal device to the IMS AGW / MRF through a media stream channel.

[0318] 1903: Upon receiving the information about the actual background object, the IMS AGW / MRF sends a request message 22 to the third party server, where the request message 22 includes information about the actual background object. As an example, the geographic location of the actual background object is used. For example, the request message 22 may be of an HTTP message type. Of course, another message type may be used instead. This is not limited in this application. For example, the request message 22 may be an HTTP request (request, req) message 22.

[0319] 1904: The third party server receives the request message 22 and obtains actual background objects based on the geographical locations of the actual background objects. The third party server sends a response message 22 to the IMS AGW / MRF. The response message 22 may include an actual background object list. The response message 22 may be of an HTTP message type. For example, the response message 22 may be an HTTP request, response message 22.

[0320] 1905: The IMS AGW / MRF obtains the actual background object from the response message 22 and renders the actual background object into the media stream 1 to obtain the AR media stream 2.

[0321] 1906: The IMS AGW / MRF sends the AR media stream 2 to the first terminal device through the media stream channel.

[0322] Based on the same inventive concept as the method embodiment, an embodiment of this application provides an apparatus, which is specifically configured to implement the method performed by an IMS core network element in the aforementioned method embodiment. The structure of the apparatus is shown in Figure 20, which includes: a receiving unit 2001, a sending unit 2002 and a processing unit 2003.

[0323] The receiving unit 2001 is configured to receive a first request message from a first terminal device, where the first request message is for requesting establishment of an auxiliary data channel, and the auxiliary data channel is for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element.

[0324] The processing unit 2003 is configured to control the establishment of an auxiliary data channel between the AR media processing network element and the first terminal device.

[0325] In a possible implementation, the processing unit 2003 controls the establishment of an auxiliary data channel between the AR media processing network element and the first terminal device via the receiving unit 2001 and the transmitting unit 2002.

[0326] The sending unit 2002 is configured to send a second request message to the AR media processing network element, where the second request message is for requesting establishment of an auxiliary data channel between the AR media processing network element and the first terminal device.

[0327] The receiving unit 2001 is further configured to receive a second response message sent by the AR media processing network element, where the second response message indicates to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel.

[0328] The transmitting unit 2002 is configured to transmit a first response message to the first terminal device, where the first response message indicates to the first terminal device that the AR media processing network element is ready to establish an auxiliary data channel.

[0329] In a possible implementation, the AR media processing network element is an IMS access gateway having AR processing capability or a media resource function MRF.

[0330] In a possible implementation, the AR media processing network element is located in the IMS network to which the first terminal device belongs, or is located in the IMS network to which the second terminal device belongs.

[0331] In a possible implementation, the first request message and the second request message carry first description parameters of the first terminal device for establishing an auxiliary data channel.

[0332] The first response message and the second response message carry second description parameters of the AR media processing network element for establishing the auxiliary data channel.

[0333] In a possible implementation, the sending unit 2002 is further configured to send an inquiry request to the AR media processing network element, the inquiry request being for inquiring whether the AR media processing network element has AR processing capability.

[0334] The receiving unit 2001 is further configured to receive an inquiry response sent by the AR media processing network element, where the inquiry response carries information indicating that the AR media processing network element has AR processing capability.

[0335] In a possible implementation, the first request message is further for requesting establishment of a media stream channel, and the media stream channel is for transmitting a media stream for transmitting content for AR communication between the first terminal device and the AR media processing network element.

[0336] The processing unit 2003 is further configured to control the establishment of a media stream channel between the AR media processing network element and the first terminal device.

[0337] In a possible implementation, the processing unit 2003 controls the establishment of a media stream channel between the AR media processing network element and the first terminal device via the sending unit 2002 and the receiving unit 2001.

[0338] The sending unit 2002 is further configured to send a second request message to the AR media processing network element, where the second request message is for requesting establishment of a media stream channel between the AR media processing network element and the first terminal device.

[0339] The receiving unit 2001 is further configured to receive a second response message sent by the AR media processing network element, where the second response message indicates to the IMS core network element that the AR media processing network element agrees to the establishment of the media stream channel.

[0340] The sending unit 2002 is further configured to send a first response message to the first terminal device, where the first response message indicates to the first terminal device that the AR media processing network element is ready to establish a media stream channel.

[0341] Based on the same inventive concept as the method embodiment, one embodiment of this application provides an apparatus that is specifically configured to implement the method performed by the AR media processing network element in the above method embodiment. The structure of the apparatus is shown in Figure 21, and includes: a receiving unit 2101, a sending unit 2102 and a processing unit 2103.

[0342] The receiving unit 2101 is configured to receive a second request message sent by an Internet Protocol Multimedia Subsystem IMS core network element, where the second request message is for requesting establishment of an auxiliary data channel between the AR media processing network element and the first terminal device, and the auxiliary data channel is for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element.

[0343] The sending unit 2102 is configured to send a second response message to the IMS core network element, where the second response message indicates to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel.

[0344] In a possible implementation form, the receiving unit 2101 is specifically configured to receive AR assistance data from the first terminal device through the auxiliary data channel after the establishment of the auxiliary data channel is completed.

[0345] In a possible implementation, a media stream channel for transmitting content for AR communication is established between the AR media processing network element and the first terminal device.

[0346] In a possible implementation, the receiving unit 2101 is configured to receive a media stream for AR communication from the first terminal device through a media stream channel.

[0347] The processing unit 2103 is configured to perform media enhancement processing on the media stream for AR communication based on the AR auxiliary data to obtain an AR media stream.

[0348] The sending unit 2102 is configured to send the AR media stream obtained through the enhanced processing to the first terminal device through a media stream channel.

[0349] In a possible implementation, the sending unit 2102 is further configured to send the AR media stream obtained by the augmented processing to a second terminal device.

[0350] In a possible implementation, the AR auxiliary data includes one or more of virtual model manipulation data, target object annotation or tracking data, facial expression or body action identification data, or pose data.

[0351] In a possible implementation, the AR media processing network element is an IMS access gateway having AR processing capability or a media resource function MRF.

[0352] In a possible implementation, the AR media processing network element is located in the IMS network to which the first terminal device belongs, or is located in the IMS network to which the second terminal device belongs.

[0353] In a possible implementation, the second request message carries a first description parameter of the first terminal device for establishing an auxiliary data channel. The processing unit 2103 is specifically configured to determine from the first description parameter a second description parameter supported by the AR media processing network element, and the second response message carries the second description parameter.

[0354] Based on the same inventive concept as the method embodiment, an embodiment of this application provides an apparatus specifically configured to implement the method performed by the terminal device in the aforementioned method embodiment. The structure of the apparatus is shown in Figure 22, which includes a receiving unit 2201, a sending unit 2202 and a processing unit 2203.

[0355] The sending unit 2202 sends a first request message to the IMS core network element, where the first request message is for requesting the establishment of an auxiliary data channel. The receiving unit 2201 receives a first response message sent by the IMS core network element, where the first response message indicates that the AR media processing network element completes the establishment of the auxiliary data channel, where the auxiliary data channel is for sending auxiliary data for AR communication between the first terminal device and the AR media processing network element.

[0356] A media stream channel for transmitting content for AR communication is established between the first terminal device and the AR media processing network element.

[0357] In a possible implementation, the sending unit 2202 sends a third request message to the IMS core network element, where the third request message is for requesting the establishment of a media stream channel. The receiving unit 2201 receives a third response message sent by the IMS core network element, where the third response message indicates to the first terminal device that the AR media processing network element is ready to establish a media stream channel.

[0358] In a possible implementation, the first request message is further for requesting the establishment of a media stream channel, and the first response message further indicates that the AR media processing network element is ready to establish the media stream channel.

[0359] In a possible implementation, the sending unit 2202 sends augmented reality AR auxiliary data to the AR media processing network element through an auxiliary data channel, and sends a media stream to the AR media processing network element through a media stream channel, where the media stream is generated during the AR communication between the first terminal device and the second terminal device, and the AR auxiliary data is used by the AR media processing network element to perform media enhancement processing on the media stream for the AR communication. The receiving unit 2201 receives the AR media stream obtained through the enhancement processing from the AR media processing network element through the media stream channel.

[0360] In a possible implementation, the receiving unit 2201 receives, via a media stream channel, an AR media stream belonging to a second terminal device and transmitted by an AR media processing network element.

[0361] In a possible implementation, the first request message carries first description parameters of the first terminal device for establishing an auxiliary data channel.

[0362] The first response message carries second description parameters of the AR media processing network element for establishing the auxiliary data channel.

[0363] In a possible implementation, the AR media processing network element is located in the IMS network to which the first terminal device belongs, or is located in the IMS network to which the second terminal device belongs.

[0364] In the embodiment of this application, the division into units is an example, and is merely a logical division of functions, and may be other divisions in actual implementation. In addition, the functional units in the embodiment of this application may be integrated into one processor, may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0365] When the embedded unit is implemented in the form of a software functional unit and sold or used as an independent product, the embedded unit may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of this application may be essentially implemented in the form of a software product, or the part that contributes to the prior art, or all or part of the technical solution may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes some instructions for instructing a terminal device (which may be a personal computer, a mobile phone, a network device, etc.) or a processor to execute all or part of the steps of the method in the embodiments of this application. The aforementioned storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0366] In the embodiments of this application, the AR media processing network element, the application server, the IMS core network element, and the terminal device may all be presented in the form of functional modules obtained through division in an integrated manner. A "module" in this specification may be an ASIC, a circuit, a processor executing one or more software or firmware programs, a memory, an integrated logic circuit, and / or another device capable of providing the aforementioned functions.

[0367] In a simple embodiment, the AR media processing network element, the application server, the IMS core network element, and the terminal device may be implemented using the structure shown in FIG.

[0368] The device 2300 shown in FIG. 23 includes at least one processor 2301 and a communication interface 2302, and may optionally further include a memory 2303.

[0369] In a possible implementation, when the AR media processing network element is implemented by using the structure shown in FIG. 23, the processor 2301 of FIG. 23 can invoke computer-executable instructions stored in memory 2303, such that the AR media processing network element can perform the method performed by the AR media processing network element in any of the method embodiments described above.

[0370] The processor 2301 may communicate with other devices via the communication interface 2302. For example, the processor 2301 receives a request message sent by an IMS core network element and sends a response message to the IMS core network element via the communication interface 2302.

[0371] Specifically, the memory 2303 stores computer-executable instructions for implementing the functions of the transmitting unit, the receiving unit, and the processing unit of Fig. 21. All the functions / implementation processes of the transmitting unit, the receiving unit, and the processing unit of Fig. 21 can be implemented by the processor 2301 of Fig. 23 in combination with the communication interface 2302 by calling the computer-executable instructions stored in the memory 2303.

[0372] In yet another possible embodiment, when the IMS core network element is implemented using the structure shown in FIG. 23, the processor 2301 of FIG. 23 can invoke computer-executable instructions stored in the memory 2303, such that the IMS core network element can perform the method performed by the IMS core network element (IMS AGW1, IMS AGW2, MRF1, or MRF2) in any of the method embodiments described above.

[0373] The processor 2301 may communicate with other devices via the communication interface 2302. For example, the processor 2301 receives messages sent by and to the AR media processing network element via the communication interface 2302.

[0374] Specifically, the memory 2303 stores computer-executable instructions for implementing the functions of the transmitting unit, the receiving unit, and the processing unit of Fig. 20. All the functions / implementation processes of the transmitting unit, the receiving unit, and the processing unit of Fig. 20 can be implemented by the processor 2301 of Fig. 23 in combination with the communication interface 2302 by calling the computer-executable instructions stored in the memory 2303.

[0375] In yet another possible implementation form, when the terminal device is implemented using the structure shown in FIG. 23, the processor 2301 of FIG. 23 may call computer-executable instructions stored in the memory 2303, such that the terminal device may perform the method performed by the first terminal device or the second terminal device in any of the aforementioned method embodiments.

[0376] The processor 2301 may communicate with other devices via the communication interface 2302. For example, the processor 2301 receives, via the communication interface 2302, an augmented media stream transmitted by an AR media processing network element, and transmits AR auxiliary data, a media stream, and the like, to the AR media processing network element.

[0377] Specifically, the memory 2303 stores computer-executable instructions for implementing the functions of the transmitting unit, the receiving unit, and the processing unit of Fig. 22. All the functions / implementation processes of the transmitting unit, the receiving unit, and the processing unit of Fig. 22 can be implemented by the processor 2301 of Fig. 23 in combination with the communication interface 2302 by calling the computer-executable instructions stored in the memory 2303.

[0378] The specific connection medium between the processor 2301 and the memory 2303 is not limited in this embodiment of the application. In this embodiment of the application, the memory 2303 is connected to the processor 2301 through a bus 2304 in the figure. The bus 2304 is represented by a thick line in the figure. The connection manner between other components is described briefly and is not limited thereto. The bus 2304 can be classified into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used to represent a bus in FIG. 23, but this does not mean that there is only one bus or only one type of bus.

[0379] Based on the above embodiment, an embodiment of this application further provides a computer storage medium. The storage medium stores a software program. When the software program is read and executed by one or more processors, the method performed by the service platform or edge computing device provided in any one or more of the above embodiments can be implemented. The computer storage medium may include any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0380] Based on the above embodiment, an embodiment of this application further provides a chip. The chip includes a processor configured to implement the function of the AR media server or the function of the terminal device or the application server in any one or more of the above embodiments. Optionally, the chip further includes a memory, and the memory is configured to store necessary program instructions and data executed by the processor. The chip may include one chip, or may include a chip and other separate devices.

[0381] It may be understood that the term "embodiment", "implementation", or "example" referred to throughout this specification means that a particular feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of this application. Thus, "in one embodiment", "in an implementation", or "in an example" appearing throughout this specification do not necessarily refer to the same embodiment. Moreover, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It may be understood that the sequence numbers of the processes described above do not imply an execution order in the embodiments of this application. The execution order of the processes should be determined based on the functions and internal logic of the processes, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0382] Furthermore, the terms "system" and "network" are generally used interchangeably in this specification. The term "and / or" in this specification describes only an association relationship for describing related objects and indicates that three relationships may exist. For example, A and / or B describes the following three cases: only A is present, both A and B are present, and only B is present. In addition, the symbol " / " in this specification generally indicates an "or" relationship between related objects. The term "at least one" in this application means one or more, i.e., including one, two, three, or more, and the term "multiple" means two or more, i.e., including two, three, or more. In addition, it can be understood that in the description of this application, terms such as "first" and "second" are merely used for distinction and explanation, and cannot be understood as indicating or implying a relative importance or as indicating or implying an order. The term "at least one of the following items" or similar expressions refers to any combination of these items, including any combination of a single item or multiple items. For example, at least one of a, b, or c may refer to a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural. In the embodiments of this application, "B corresponding to A" may be understood to indicate that B is associated with A and B may be determined according to A. However, it may be further understood that determining B based on A does not mean that B is determined based only on A. B may alternatively be determined based on A and / or other information. In addition, the terms "include" and "have" in the embodiments, claims, and accompanying drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the enumerated steps or modules, and may further include unenumerated steps or modules.

[0383] It may be understood that the processor in the embodiments of this application may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0384] The method steps in the embodiments of this application may be implemented in a hardware manner or in a manner of executing software instructions by a processor. The software instructions may include corresponding software modules. The software modules may be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the art. For example, the storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may be a component of the processor. The processor and the storage medium may be located in an ASIC. In addition, the ASIC may be located in a network device or a terminal device. Of course, the processor and the storage medium may exist as separate components in a network device or a terminal device.

[0385] All or part of the above-mentioned embodiments may be implemented by software, hardware, firmware, or any combination thereof. When software is used to implement the embodiments, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded into a computer and executed, the procedures or functions of the embodiments of this application are executed completely or partially. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer programs or instructions may be stored in a computer-readable storage medium or transmitted by using a computer-readable storage medium. The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device such as a server that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape, an optical medium, such as a DVD, or a semiconductor medium, such as a solid state disk (SSD).

[0386] In various embodiments of this application, unless otherwise specified or there is no logical contradiction, the terms and / or descriptions between different embodiments are consistent and may be cross-referenced, and the technical features of different embodiments may be combined based on their internal logical relationships to form a new embodiment. [Explanation of symbols]

[0387] 2001 Receiver Unit 2002 Sending Unit 2003 Processing Unit 2101 Receiver unit 2102 Transmitter unit 2103 Processing Unit 2201 Receiver Unit 2202 Transmitter Unit 2203 Processing Unit 2300 equipment 2301 Processor 2302 Communication Interface 2303 Memory 2304 Bus

Claims

1. An augmented reality (AR) communication system, comprising: an Internet Protocol Multimedia Subsystem (IMS) core network element; and a first AR media processing network element, the IMS core network element is configured to control establishment of a first media stream channel between the first AR media processing network element and a first terminal device and an auxiliary data channel between the first AR media processing network element and a first terminal device; the first AR media processing network element is configured to receive first AR auxiliary data from the first terminal device through the established auxiliary data channel and to receive a first media stream for AR communication between the first terminal device and a second terminal device through the established media stream channel; The first AR media processing network element is further configured to perform enhanced processing on the media stream based on the first AR auxiliary data to obtain a first AR media stream, and transmit the first AR media stream to the first terminal device through the media stream channel. Augmented reality communication system.

2. The system of claim 1 , wherein the first AR media processing network element is an IMS access gateway having an AR processing capability or a media resource function (MRF).

3. The system according to claim 1 or 2, wherein the first AR media processing network element is located in an IMS network to which the first terminal device belongs.

4. The system further comprises a second AR media processing network element located in an IMS network to which the second terminal device belongs; the first AR media processing network element is further configured to transmit the AR media stream to the second AR media processing network element; the second AR media processing network element is configured to receive the AR media stream transmitted by the first AR media processing network element and transmit the AR media stream to the second terminal device; The system of claim 3.

5. the first AR media processing network element is located in an IMS network to which the second terminal device belongs, the system further comprising a third AR media processing network element located in the IMS network to which the first terminal device belongs, the third AR media processing network element having no AR processing capability, the media stream channel between the first AR media processing network element and the first terminal device comprising a first media stream channel between the first AR media processing network element and the third AR media processing network element and a second media stream channel between the third AR media processing network element and the first terminal device; the third AR media processing network element is configured to receive the media stream from the first terminal device through the second media stream channel and forward the media stream to the first AR media processing network element through the first media stream channel, and is configured to receive the AR media stream from the first AR media processing network element through the first media stream channel and transmit the AR media stream to the first terminal device through the second media stream channel; 3. A system according to claim 1 or 2.

6. the first AR media processing network element is further configured to receive target object information from the first terminal device through the auxiliary data channel; The AR media processing network element is further configured to obtain, from a third party server, a target object corresponding to the target object information. A system according to any one of claims 1 to 5.

7. the target object is a virtual object or an identifier of a virtual object; the first AR media processing network element is further configured to transmit the virtual object or the identifier of the virtual object to the first terminal device through the auxiliary data channel; The system of claim 6.

8. The target object is a real background object, and the first AR media processing network element is further configured to receive, through the media stream channel, a second media stream generated during an AR communication and transmitted by the first terminal device; The AR media processing network element is further configured to render and combine the real background object and the second media stream to obtain a second AR media stream, and transmit the second AR media stream to the first terminal device through the media stream channel. The system of claim 6.

9. An application server is further provided, the application server is configured to receive a first request message from the AR media processing network element and forward the first request message to the third party server, and to receive a first response message from the third party server and forward the first response message to the first AR media processing network element; The system of claim 6.

10. the application server is configured with an address of the third party server; the application server is specifically configured to forward the first request message to the third party server based on the address of the third party server, and forward a second response message to the AR media processing network element; The system of claim 9.

11. 10. The system of claim 9, wherein the application server is further configured to send an address of the third party server to the first AR media processing network element in the procedure of establishing the auxiliary data channel between the first terminal device and the first AR media processing network element.

12. receiving, by an Internet Protocol Multimedia Subsystem (IMS) core network element, a first request message from a first terminal device, the first request message being for requesting establishment of an auxiliary data channel, the auxiliary data channel being for transmitting auxiliary data for AR communication between the first terminal device and an AR media processing network element; controlling, by the IMS core network element, establishment of the auxiliary data channel between the AR media processing network element and the first terminal device; An augmented reality (AR) communication method comprising:

13. The step of controlling, by the IMS core network element, the establishment of the auxiliary data channel between the AR media processing network element and the first terminal device, comprises: sending, by the IMS core network element, a second request message to the AR media processing network element, the second request message for requesting establishment of the auxiliary data channel between the AR media processing network element and the first terminal device; receiving, by the IMS core network element, a second response message sent by the AR media processing network element, the second response message indicating to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel; sending a first response message by the IMS core network element to the first terminal device, the first response message indicating to the first terminal device that the AR media processing network element is ready to establish the auxiliary data channel; 13. The method of claim 12, comprising:

14. The method according to claim 12 or 13, wherein the AR media processing network element is an IMS access gateway having an AR processing capability or a media resource function MRF.

15. The method according to claim 12 or 13, wherein the AR media processing network element is located in an IMS network to which the first terminal device belongs or is located in an IMS network to which a second terminal device belongs.

16. the first request message and the second request message convey first description parameters of the first terminal device for establishing the auxiliary data channel; the first response message and the second response message convey second description parameters of the AR media processing network element for establishing the auxiliary data channel; The method of claim 13.

17. The method comprises: sending an inquiry request by the IMS core network element to the AR media processing network element, the inquiry request being for inquiring whether the AR media processing network element has AR processing capability; receiving, by the IMS core network element, an inquiry response sent by the AR media processing network element, the inquiry response carrying information indicating that the AR media processing network element has the AR processing capability; 17. The method of any one of claims 13 to 16, further comprising:

18. The method comprises: controlling, by the IMS core network element, establishment of a media stream channel between the AR media processing network element and the first terminal device, the media stream channel for transmitting a media stream for transmitting content for AR communication between the first terminal device and the AR media processing network element; The method of claim 12, further comprising:

19. receiving, by the IMS core network element, a third request message from the first terminal device, the third request message being for further requesting establishment of the media stream channel; The step of controlling, by the IMS core network element, the establishment of a media stream channel between the AR media processing network element and the first terminal device, comprises: sending, by the IMS core network element, a fourth request message to the AR media processing network element, the fourth request message for requesting establishment of the media stream channel between the AR media processing network element and the first terminal device; receiving, by the IMS core network element, a fourth response message sent by the AR media processing network element, the fourth response message indicating to the IMS core network element that the AR media processing network element agrees to the establishment of the media stream channel; sending, by the IMS core network element, a third response message to the first terminal device, the third response message indicating to the first terminal device that the AR media processing network element is ready to establish the media stream channel; 20. The method of claim 18, further comprising:

20. receiving, by an AR media processing network element, a second request message sent by an Internet Protocol Multimedia Subsystem (IMS) core network element, the second request message for requesting establishment of an auxiliary data channel between the AR media processing network element and a first terminal device, the auxiliary data channel for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element; sending, by the AR media processing network element, a second response message to the IMS core network element, the second response message indicating to the IMS core network element that the AR media processing network element agrees to the establishment of the auxiliary data channel; An augmented reality (AR) communication method comprising:

21. The method comprises: receiving, by the AR media processing network element, AR assistance data from the first terminal device over the auxiliary data channel after the establishment of the auxiliary data channel is completed; 21. The method of claim 20, further comprising:

22. The method comprises: establishing a media stream channel between the AR media processing network element and the first terminal device, the media stream channel being for transmitting content for AR communication; 22. The method of claim 20 or 21, further comprising:

23. The method comprises: receiving, by the AR media processing network element, a media stream for the AR communication from the first terminal device over a media stream channel; performing, by the AR media processing network element, media enhancement processing on the media stream for the AR communication based on the AR auxiliary data to obtain an AR media stream; transmitting, by the AR media processing network element, the AR media stream obtained through augmented processing to the first terminal device through the media stream channel; 22. The method of claim 21, further comprising:

24. The method comprises: transmitting, by the AR media processing network element, the AR media stream obtained through augmented processing to a second terminal device; 24. The method of claim 23, further comprising:

25. The method of claim 21 , wherein the AR auxiliary data includes one or more of virtual model manipulation data, target object annotation or tracking data, facial expression or body action identification data, or pose data.

26. 26. The method according to claim 20, wherein the AR media processing network element is an IMS access gateway having an AR processing capability or a media resource function MRF.

27. The method of claim 24, wherein the AR media processing network element is located in an IMS network to which the first terminal device belongs or is located in an IMS network to which the second terminal device belongs.

28. The second request message carries first description parameters of the first terminal device for establishing the auxiliary data channel, and the method includes: determining, by the AR media processing network element, from the first description parameters, second description parameters supported by the AR media processing network element; the second response message conveys the second description parameters; 28. The method according to any one of claims 20 to 27.

29. sending a first request message by a first terminal device to an IMS core network element, the first request message being for requesting establishment of an auxiliary data channel; receiving, by the first terminal device, a first response message sent by the IMS core network element, the first response message indicating that an AR media processing network element completes the establishment of the auxiliary data channel, the auxiliary data channel being for transmitting auxiliary data for AR communication between the first terminal device and the AR media processing network element; An augmented reality (AR) communication method comprising:

30. The method comprises: establishing a media stream channel between the first terminal device and the AR media processing network element for transmitting content for the AR communication; 30. The method of claim 29, further comprising:

31. The step of establishing a media stream channel for transmitting content for the AR communication between the first terminal device and the AR media processing network element includes: sending, by the first terminal device, a third request message to the IMS core network element, the third request message for requesting establishment of the media stream channel; receiving, by the first terminal device, a third response message sent by the IMS core network element, the third response message indicating to the first terminal device that the AR media processing network element is ready to establish the media stream channel; 31. The method of claim 29 or 30, comprising:

32. 30. The method of claim 29, wherein the first request message is for further requesting establishment of a media stream channel, and the first response message further indicates that the AR media processing network element is ready to establish the media stream channel.

33. The method comprises: transmitting, by the first terminal device, augmented reality (AR) auxiliary data through the auxiliary data channel and a media stream through the media stream channel to the AR media processing network element, the media stream being generated during an AR communication between the first terminal device and a second terminal device, and the AR auxiliary data being used by the AR media processing network element to perform media enhancement processing on the media stream for the AR communication; receiving, by the first terminal device, an AR media stream obtained through augmented processing from the AR media processing network element through the media stream channel; 33. The method of any one of claims 30 to 32, further comprising:

34. The method comprises: receiving, by the first terminal device, through the media stream channel, an AR media stream belonging to the second terminal device and transmitted by the AR media processing network element; 34. The method of claim 33, further comprising:

35. the first request message conveys first description parameters of the first terminal device for establishing the auxiliary data channel; the first response message conveys second description parameters of the AR media processing network element for establishing the auxiliary data channel; 35. The method of any one of claims 29 to 34.

36. The method of claim 34, wherein the AR media processing network element is located in an IMS network to which the first terminal device belongs or is located in an IMS network to which the second terminal device belongs.

37. A communication device, the device comprising a communication interface and a processor; the communication interface is configured to transmit and receive signals; The processor is configured to transmit and receive signals through the communication interface and to implement a method according to any one of claims 12 to 19. Communications equipment.

38. A communication device, the device comprising a communication interface and a processor; the communication interface is configured to transmit and receive signals; The processor is configured to transmit and receive signals through the communication interface and to implement a method according to any one of claims 20 to 28. Communications equipment.

39. A communication device, the device comprising a communication interface and a processor; the communication interface is configured to transmit and receive signals; The processor is configured to transmit and receive signals through the communication interface and to implement a method according to any one of claims 29 to 36. Communications equipment.

Images