Media negotiation method, apparatus, device, medium, and program product

Abstract

This application discloses a media negotiation method, apparatus, device, medium, and program product, relating to the field of communication technology. The method, applied to a first terminal, includes: initiating a first session to a second terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability; reporting the second expected media capability to a server if the first expected media capability does not match a second expected media capability sent by the second terminal; receiving first indication information sent by the server, the first indication information indicating a first media negotiation method, the first media negotiation method being determined based on the second expected media capability; and initiating a second session to the second terminal using the first media negotiation method. This application's embodiments can improve the user experience when media negotiation results are uncoordinated or mismatched, resulting in the inability to establish a session.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a media negotiation method, apparatus, device, medium, and program product. Background Technology

[0002] The Session Description Protocol (SDP) is a text-based protocol used to describe session information in multimedia sessions. In Web Real-Time Communications (WebRTC), SDP is used to describe the media parameters between the communicating parties. By exchanging SDP information, the two parties can negotiate optimal communication parameters to ensure correct communication. However, in related technologies, the negotiated SDP protocol results can only satisfy the minimum user experience for both parties. There may be no overlap or mismatch in the negotiated results, leading to session failure and a poor user experience. Summary of the Invention

[0003] This application provides a media negotiation method, apparatus, device, medium, and program product to solve the problem in related technologies where media negotiation results are not overlapping or mismatched, resulting in the inability to establish a session and a poor user experience.

[0004] In a first aspect, embodiments of this application provide a media negotiation method applied to a first terminal, the method comprising:

[0005] A first session is initiated to the second terminal. The first session carries first media capability information, which includes a first expected media capability.

[0006] If the first expected media capability does not match the second expected media capability sent by the second terminal, the second expected media capability shall be reported to the server.

[0007] Receive first indication information sent by the server, the first indication information being used to indicate a first media negotiation method, the first media negotiation method being determined based on the second expected media capability;

[0008] The second session is initiated to the second terminal using the first media negotiation method.

[0009] Secondly, embodiments of this application also provide a media negotiation method applied to a server, the method comprising:

[0010] The system receives first media capability information and second media capability information sent by a first terminal and a second terminal, respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability.

[0011] The first media negotiation method is determined based on the second expected media capability, and the second media negotiation method is determined based on the first expected media capability information;

[0012] Send a first instruction message to the first terminal, the first instruction message being used to indicate the first media negotiation method; and send a second instruction message to the second terminal, the second instruction message being used to indicate the second media negotiation method.

[0013] Thirdly, this application also provides a media negotiation method applied to a second terminal, the method comprising:

[0014] Receive a first session initiated by a first terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability;

[0015] Send a second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal;

[0016] If the first expected media capability and the second expected media capability do not match, the first expected media capability shall be reported to the server.

[0017] Receive a second indication information sent by the server, the second indication information being used to indicate a second media negotiation method, the second media negotiation method being determined based on the first expected media capability;

[0018] The second session initiated by the first terminal is responded to using the second media negotiation method.

[0019] Fourthly, embodiments of this application also provide a media negotiation device applied to a first terminal, the device comprising:

[0020] The first session module is used to initiate a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability.

[0021] The first reporting module is used to report the second expected media capability to the server when the first expected media capability does not match the second expected media capability sent by the second terminal.

[0022] A first receiving module is configured to receive first indication information sent by the server, the first indication information being used to indicate a first media negotiation method, the first media negotiation method being determined based on the second expected media capability;

[0023] The second session module is used to initiate a second session with the second terminal using the first media negotiation method.

[0024] Fifthly, embodiments of this application provide a media negotiation apparatus applied to a server, the apparatus comprising:

[0025] The second receiving module is used to receive first media capability information and second media capability information sent by the first terminal and the second terminal respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability.

[0026] The negotiation determination module is used to determine a first media negotiation method based on the second expected media capability, and to determine a second media negotiation method based on the first expected media capability information;

[0027] The first sending module is configured to send first indication information to the first terminal, the first indication information being used to indicate the first media negotiation mode, and to send second indication information to the second terminal, the second indication information being used to indicate the second media negotiation mode.

[0028] Sixthly, embodiments of this application provide a media negotiation device applied to a second terminal, the device comprising:

[0029] The third receiving module is used to receive a first session initiated by the first terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability;

[0030] The second sending module is used to send a second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal;

[0031] The second reporting module is used to report the first expected media capability to the server when the first expected media capability and the second expected media capability do not match.

[0032] The fourth receiving module is used to receive second indication information sent by the server, the second indication information being used to indicate a second media negotiation method, the second media negotiation method being determined based on the first expected media capability;

[0033] The session response module is used to respond to the second session initiated by the first terminal using the second media negotiation method.

[0034] In a seventh aspect, embodiments of this application also provide a communication device, including: a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor;

[0035] The transceiver is configured to implement the steps of the method as described in any one of the first aspects; or implement the steps of the method as described in any one of the second aspects; or implement the steps of the method as described in any one of the third aspects.

[0036] Eighthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any one of the first aspects; or implements the steps of the method as described in any one of the second aspects; or implements the steps of the method as described in any one of the third aspects.

[0037] In a ninth aspect, embodiments of this application also provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method as described in any one of the first aspects; or implement the steps of the method as described in any one of the second aspects; or implement the steps of the method as described in any one of the third aspects.

[0038] In this embodiment, the first terminal carries first media capability information in the first session initiated to the second terminal, and the first media capability information includes the first terminal's first expected media capability. Subsequently, it is determined whether the first expected media capability matches the second expected media capability of the second terminal. If they do not match, the first terminal reports the second expected media capability to the server. The server then determines the first media negotiation method that the first terminal can use when communicating with the second terminal, thus implementing a renegotiation process between the first and second terminals, and enabling a session between them. In this way, this embodiment can reduce the probability of session failure due to differences in media capabilities when the first and second terminals use different media capabilities, optimizing session services and improving the user's communication session experience. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1This is a flowchart of a media negotiation method applied to a first terminal, provided in an embodiment of this application;

[0041] Figure 2 This is one of the flowcharts of a media negotiation method provided in the embodiments of this application;

[0042] Figure 3 This is a second flowchart of a media negotiation method provided in the embodiments of this application;

[0043] Figure 4 This is the third flowchart of a media negotiation method provided in the embodiments of this application;

[0044] Figure 5 This is the fourth flowchart of a media negotiation method provided in the embodiments of this application;

[0045] Figure 6 This is a flowchart of a media negotiation method applied to a server, provided in an embodiment of this application;

[0046] Figure 7 This is a flowchart of a media negotiation method applied to a second terminal, provided in an embodiment of this application;

[0047] Figure 8 This is one of the structural schematic diagrams of a media negotiation device provided in the embodiments of this application;

[0048] Figure 9 This is a second schematic diagram of the structure of a media negotiation device provided in the embodiments of this application;

[0049] Figure 10 This is the third schematic diagram of a media negotiation device provided in the embodiments of this application;

[0050] Figure 11 This is one of the structural diagrams of a communication device provided in the embodiments of this application;

[0051] Figure 12 This is a second structural diagram of a communication device provided in an embodiment of this application;

[0052] Figure 13 This is the third structural diagram of a communication device provided in the embodiments of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] The following description, in conjunction with the accompanying drawings, details a media negotiation method, apparatus, device, medium, and program product provided in this application through specific embodiments and application scenarios.

[0055] See Figure 1 , Figure 1 This is one of the flowcharts of a media negotiation method provided in the embodiments of this application, applied to a first terminal, such as... Figure 1 As shown, it includes the following steps:

[0056] Step 101: Initiate a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability.

[0057] It should be noted that the first terminal in this application embodiment can be any terminal device used by the calling user, such as a mobile phone, tablet computer, laptop computer, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, vehicle-mounted device (VUE), pedestrian terminal (PUE), smart home (home devices with wireless communication capabilities), game console, personal computer, etc. Wearable devices include: smartwatches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart necklaces, smart anklets, smart wristbands, etc.), smart clothing, etc. The second terminal can be the terminal device used by the called user; specific types are described in the relevant embodiments of the first terminal. The first terminal acts as the calling user, and the second terminal acts as the called user initiating a session. The calling user initiates the session to the called user, and the calling and called users can adaptively perform media negotiation processing based on their actual media communication capabilities.

[0058] In the above steps, the first terminal, as the calling user, directly initiates the first session to the second terminal. During the process of initiating the first session, the first terminal and the second terminal will negotiate the first session description protocol (SDP) to determine whether the media expected by both parties are matched. Then, based on the matching result, it is convenient to determine whether to conduct a session or to renegotiation of the media.

[0059] It is understandable that the aforementioned first expected media capability can be the media capability that the communication method expected to be used in the first session initiated by the first terminal should possess. For example, if the first terminal, as the calling user, initiates a panoramic video call to the second terminal, the first expected media capability could be both panoramic video decoding capability and panoramic video decoding capability, or the first expected media capability could only be panoramic video decoding capability. That is, it requires that the first terminal be able to decode the panoramic video sent by the second terminal during the panoramic video call, so that the panoramic video can be displayed on the first terminal's display interface.

[0060] Step 102: If the first expected media capability does not match the second expected media capability sent by the second terminal, report the second expected media capability to the server.

[0061] In this embodiment, the ability of the first and second terminals to continue their conversation can be determined by judging whether the first expected media capability matches the second expected media capability, or by renegotiating the media through the server. Specifically, when the first and second expected media capabilities match, it indicates that the first and second terminals each meet the session expectations, and the first session can be established for communication. For example, if the first expected media capability is a panoramic decoding capability and the second expected media capability is also a panoramic decoding capability, and the first and second expected media capabilities match, then a panoramic video call can be established between the first and second terminals. As another example, when the first expected media capability is a front-facing camera-related capability and / or a rear-facing camera-related capability, and the second expected media capability is also a front-facing camera-related capability and / or a rear-facing camera-related capability, the first and second expected media capabilities match, and a regular video call can be directly established between the first and second terminals.

[0062] Please refer to Figure 2 , Figure 2 This is one of the flowcharts of a media negotiation method provided in the embodiments of this application, which specifically includes the following steps:

[0063] Step 201: The first terminal initiates a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability. The second terminal receives the first session initiated by the first terminal. The first session carries first media capability information, which includes a first expected media capability.

[0064] Step 202: Determine whether the first expected media capability matches the second expected media capability;

[0065] Step 203: If the first expected media capability matches the second expected media capability, the first terminal and the second terminal establish a session.

[0066] Step 204: If the first expected media capability and the second expected media capability do not match, the first terminal and the second terminal shall respectively report the first expected media capability and the second expected media capability to the server.

[0067] Step 205: The server determines the first media negotiation method based on the second expected media capability, and determines the second media negotiation method based on the first expected media capability information;

[0068] Step 206: The first terminal uses the first media negotiation method and the second terminal uses the second media negotiation method to renegotiate the media. The next step is to execute step 203.

[0069] Furthermore, when the first expected media capability and the second expected media capability do not match, for example, if the first expected media capability is a front-facing camera-related capability and / or a rear-facing camera-related capability, and the second expected media capability is a panoramic decoding capability, it indicates that the first terminal expects to communicate via the front-facing camera-related capability or the rear-facing camera-related capability, while the second terminal expects to receive service calls via panoramic video. In this case, the first and second expected media capabilities do not match, and the first and second terminals cannot directly establish a session. As another example, if the first expected media capability is a panoramic decoding capability, and the second expected media capability is a front-facing camera-related capability or a rear-facing camera-related capability, then the second terminal cannot respond to the first terminal's panoramic request via the front-facing camera-related capability or the rear-facing camera-related capability; in this case, the first and second expected media capabilities do not match.

[0070] In this scenario, media negotiation can be performed by setting up a server between the first and second terminals. The server acts as a media processing platform, capable of orchestrating and constructing the media negotiation methods (i.e., the communication methods for the session) that the second terminal can adopt based on the first terminal's expected media capabilities, and also capable of orchestrating and constructing the media negotiation methods that the first terminal can adopt based on the second terminal's second media capabilities. Therefore, the first terminal can report its second expected media capabilities to the server, allowing the server to determine the appropriate media negotiation methods. Similarly, the second terminal can also report the first terminal's first expected media capabilities to the server, allowing the server to determine the appropriate media negotiation methods, thus enabling a session between the first and second terminals.

[0071] Step 103: Receive the first indication information sent by the server. The first indication information is used to indicate the first media negotiation method, which is determined based on the second expected media capability.

[0072] It is worth mentioning that the server in this embodiment can be a media negotiation platform, including a Session Border Controller (SBC) and a media processing platform. The SBC is used to receive messages sent by the first terminal and the second terminal, and also to send messages to the first terminal and the second terminal. The media processing platform is used to determine the appropriate media negotiation method for the first terminal and the second terminal.

[0073] In one specific embodiment, the server receives a second expected media capability sent by a first terminal and a first expected media capability sent by a second terminal. Combining this with a pre-collected first capability set and a second capability set, the server determines a first media negotiation method and a second media negotiation method. Then, the server sends instruction information carrying the first and second media negotiation methods to both the first and second terminals respectively; alternatively, the server sends a first instruction information and a second instruction information to both the first and second terminals respectively, with the first instruction information instructing the first terminal to use the first media negotiation method and the second instruction information instructing the second terminal to use the second media negotiation method.

[0074] It is understandable that when the first expected media capability is panoramic decoding capability and the second expected media capability is front-facing camera-related capability or rear-facing camera-related capability, the server can determine the first media negotiation method as instructing the first terminal to change the first expected media capability to front-facing camera-related capability or rear-facing camera-related capability, and determine the second media negotiation method as instructing the second terminal to respond to the second session initiated by the first terminal.

[0075] In another embodiment, when the server's first expected media capability is either a front-facing or rear-facing camera-related capability, and the second expected media capability is a panoramic decoding capability, the first media negotiation method can be to instruct the first terminal to perform a media update, updating the first expected media capability to panoramic decoding capability, requesting multiple media paths, and establishing a session channel between the server and the first terminal. The second media negotiation method can be to instruct the second terminal to respond to a second session initiated by the first terminal. The session channel between the server and the first terminal can transmit the panoramic video synthesized on the server to the first terminal, at which point the first terminal receives the panoramic video.

[0076] Step 104: Initiate a second session with the second terminal using the first media negotiation method.

[0077] According to the aforementioned embodiments, when the first negotiation method instructs the first terminal to change the first expected media capability from panoramic decoding capability to front-facing camera-related capability or rear-facing camera-related capability, the first terminal can initiate a session with the second terminal again according to the first media negotiation method. At this time, the second expected media capability is either front-facing camera-related capability or rear-facing camera-related capability, indicating that the first expected media capability and the second expected media capability match, and a second session can be directly established. Since the first terminal initially expected both parties to communicate via panoramic video, the first terminal can achieve panoramic video communication (achieved through panoramic decoding capability), and the second terminal can achieve front-facing camera video or rear-facing camera video communication (achieved through front-facing camera-related capability and / or rear-facing camera-related capability).

[0078] Furthermore, if the first negotiation method involves instructing the first terminal to update its first expected media capability to panoramic decoding capability and requesting multiple media paths to establish a session channel between the server and the first terminal, then the first terminal initiates a second session with the second terminal. In this case, the second expected media capability is panoramic decoding capability, and the first and second expected capabilities match, thus establishing the second session between the first and second terminals. Simultaneously, the first terminal can receive panoramic video through the session channel, and the second terminal can also receive panoramic video, allowing the calling and called users to communicate via panoramic video.

[0079] In one embodiment of the media negotiation method in this application, it can be specifically applied to communication session services. A server generates the expected media capabilities and capability sets for both the first and second terminals, orchestrates new expected media capabilities, and updates the session to ensure the continuity of media negotiation. The first and second media negotiation methods determined in the same embodiment of this application are corresponding and are methods that can satisfy the maximum media capability requirement (union of media capabilities). By determining the maximum set of media capabilities for both the first and second terminals, this embodiment uses the media method most desired by the terminal user for communication, effectively improving the user's video call experience.

[0080] Optionally, the first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities;

[0081] The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

[0082] In some embodiments, the session initiated by the first terminal as the calling user to the second terminal may carry first media capability information. The first media capability information may include its own capability set (i.e., the first capability set) and a first expected media capability. The first capability set may be the media capabilities that the first terminal itself possesses. The first terminal may be a device equipped with a front-facing camera and a rear-facing camera. Since it does not have panoramic video, image, or other acquisition accessories and does not have panoramic encoding capabilities, the first capability set of the first terminal may include at least one of the following: front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities.

[0083] Among these, front-facing camera-related capabilities can be front-facing camera encoding and decoding capabilities, while rear-facing camera-related capabilities can be rear-facing camera encoding and decoding capabilities. The first expected media capability can be the capability required by the first terminal for the communication method it expects to perform with the second terminal in the initiated session. The first expected media capability can be any one of front-facing camera encoding, rear-facing camera encoding, or panoramic decoding capabilities. For example, if the first terminal expects to conduct a panoramic video call, then the first expected media capability could be panoramic decoding capabilities, etc. As another example, if the first terminal expects to conduct a regular video call, then the first expected media capability could be either a front-facing camera-related capability or a rear-facing camera-related capability.

[0084] Similarly, when the second terminal receives the first session, which carries a first expected media capability, the second terminal can send a response message to the first terminal according to its own needs. This response message can carry a second expected media capability. For example, if the second terminal expects to conduct the session via panoramic video, then the second expected media capability may include panoramic output capability and panoramic decoding capability. The second terminal's own media capability set can be a second capability set, specifically including at least one of the following: front-facing encoding capability, front-facing decoding capability, rear-facing encoding capability, rear-facing decoding capability, panoramic encoding capability, and panoramic decoding capability. This application embodiment does not limit the types or number of media capabilities included in the capability sets of the first and second terminals.

[0085] It can be understood that the panoramic encoding capability in this application embodiment means that the terminal has the ability to decode acquired videos, images, etc., and output them in the form of panoramic videos, panoramic images, etc. Similarly, the panoramic encoding capability in this application embodiment means that the terminal has the ability to decode panoramic videos and panoramic images received from other terminals and display them on its display screen.

[0086] In some embodiments, the first terminal and the second terminal may possess all available media capabilities and optional expected media capabilities. As shown in Table 1, Table 1 lists the complete capability set and expected media capabilities of the first terminal and the second terminal.

[0087] Table 1

[0088]

[0089]

[0090] In some other embodiments of this application, the capability set may differ due to limitations of certain terminal accessories. For example, when the first terminal is a mobile phone, its first capability set may include front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. When the first terminal is a panoramic camera or virtual reality (VR) glasses with communication functions, its first capability set may include front-facing camera encoding capabilities, front-facing camera decoding capabilities, rear-facing camera encoding capabilities, rear-facing camera decoding capabilities, panoramic encoding capabilities, and panoramic decoding capabilities.

[0091] It is understood that subsequent embodiments of this application may be carried out under the capability sets and expected media capability settings of the first and second terminals in List 2 below.

[0092] Table 2

[0093]

[0094] In this way, by determining the capability sets possessed by different terminals, the server can provide an appropriate media negotiation method for the first and second terminals based on their actual and expected media capabilities, thereby optimizing the communication process and services.

[0095] Optionally, when the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0096] The step of reporting the second expected media capability to the server includes:

[0097] Send a session error message to the server, the session error message including the second expected media capability, and the session error message being used to indicate that the negotiation between the first terminal and the second terminal failed;

[0098] The first indication information is carried in the first update message, which is used to request the first terminal to establish multiple media paths. The multiple media paths include media paths based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0099] It is understandable that different terminals may experience session failures due to their varying media capabilities. To address this, a media processing platform can be used for media negotiation between the first and second terminals that need to conduct a session. The media processing platform organizes the media capability information reported by the first and second terminals and constructs a maximum set of media capabilities, so that the first and second terminals can adaptively perform media negotiation processing according to actual communication needs.

[0100] In some specific embodiments, please refer to Figure 3 As shown, Figure 3 This is a flowchart illustrating a media negotiation method in one of the embodiments of this application, specifically including the following steps:

[0101] a11. The first terminal initiates the first session, and the first expected media capability carried in the first session is the panoramic decoding capability of the first terminal.

[0102] It should be noted that the first expected media capability in this step indicates the communication method that the first terminal expects the first session to adopt. For example, the first expected media capability is panoramic decoding capability. However, due to the configuration limitations of the first terminal itself, its first capability set includes the front-facing camera-related capability (Video1-front) and the rear-facing camera-related capability (Video2-back), but does not contain panoramic encoding capability and does not support panoramic video encoding.

[0103] a12. The second terminal receives the first session initiated by the first terminal and sends the second expected media capability to the first terminal.

[0104] Specifically, when the second terminal receives the first session, it selects panoramic video call to answer based on its expectations and its own second set of capabilities, and informs the first terminal that it has selected panoramic video call to answer, that is, it sends the second expected media capability to the first terminal.

[0105] a13. The first terminal sends a session error message to the server's SBC, the session error message including the second expected media capability.

[0106] In this step, since the first terminal's capability set only includes front-facing and rear-facing camera-related capabilities and does not support encoding panoramic video, the first and second expected media capabilities do not match in this negotiation, resulting in negotiation failure. At this time, the first terminal sends a session error message to the server, which is received by the server's SBC. The session error message can be a 4XX message, indicating a client error, meaning the negotiation failed, and indicating that the negotiation between the first and second terminals failed. Furthermore, the session error message can also carry the second expected media capability of the second terminal, so that the media processing platform on the server can determine the first media negotiation method based on the second expected media capability.

[0107] a14. SBC sends session error messages to the media processing platform on the server.

[0108] a15. The media processing platform sends the first update message to the SBC;

[0109] a16. SBC sends a first update message to the first terminal. The first update message is used to request the first terminal to establish a multi-channel media path.

[0110] In a specific embodiment of this application, after receiving a session error message from the first terminal, the media processing platform negotiates media with the first terminal by sending a first update message. The first instruction information is carried in the first update message, instructing the first terminal to perform a media update and requesting the first terminal to establish multiple media channels. Specifically, the multiple media channels can be channels for transmitting multiple video media.

[0111] a17. When the first terminal establishes the multi-channel media path, the first terminal sends a first response message to the server;

[0112] Furthermore, the first terminal establishes a multimedia channel with the server based on the first instruction information, and then the first terminal sends a first response message to the server. The first response message can inform the server that the multimedia channel has been established and that it can communicate with the second terminal through this.

[0113] a18. The server and the first terminal establish a first session channel based on multiple media paths;

[0114] It should be understood that the first session channel can be a channel that can transmit various media resources between the first terminal and the server, so that the server can obtain the video resources collected by the first terminal through the first session channel and then perform panoramic synthesis.

[0115] a19. The media processing platform in the server acquires at least one piece of first information based on the first session channel and the first terminal's front-facing and rear-facing camera-related capabilities; and synthesizes at least one piece of first information to obtain panoramic composite information.

[0116] In a specific embodiment, the first session channel can transmit first information collected by the first terminal based on its front-facing and rear-facing camera capabilities to the server. This first information can be audio, video, images, or other similar data. The media processing platform on the server can integrate multiple pieces of first information and synthesize them into a single panoramic image or video to obtain panoramic composite information. After completing the panoramic synthesis, the panoramic composite information is then sent to the first terminal through the first session channel. This allows the first terminal to display the panoramic composite information on its interface even if it lacks panoramic encoding capabilities, improving the user's communication experience and optimizing session services.

[0117] a20. The server sends the sixth update message to the second terminal;

[0118] In addition, the server can also send a sixth update message to the second terminal, wherein the second indication information is carried in the sixth update message. The sixth update message can instruct the second terminal to respond to the second session initiated by the first terminal based on the first session channel.

[0119] a21. The first terminal and the second terminal conduct a second session.

[0120] In this embodiment, the first terminal lacks panoramic encoding capabilities due to a deficiency in its first capability set, resulting in a mismatch between the first expected media capability and the second expected media capability. Therefore, the server needs to renegotiate media between the first and second terminals. In this way, the server can send a first instruction message to the first terminal, instructing it to update its first expected media capability and establishing a first session channel with it. This allows the server to meet the panoramic video expectations of the first terminal even when it lacks panoramic encoding capabilities, enabling panoramic video communication between the first and second terminals, optimizing the service session process, meeting the communication needs of end users, and improving the end user communication experience.

[0121] Optionally, initiating a second session with the second terminal using the first media negotiation method includes:

[0122] When the first terminal establishes the multi-channel media path, a first response message is sent to the server, and a first session channel based on the multi-channel media path is established with the server;

[0123] Initiate a second session to the second terminal.

[0124] Please continue to refer to this. Figure 3In some embodiments, the first terminal responds to the first instruction information and continues the second session with the second terminal using a first media negotiation method. Specifically, the first media negotiation method may involve the first terminal updating its first expected media capabilities and establishing a first session channel with the server to facilitate matching with the second expected media capabilities, and to transmit the panoramic composite information synthesized by the server through the first session channel.

[0125] Thus, in the second session between the first terminal and the second terminal, the second terminal can receive and acquire panoramic video based on its panoramic encoding and decoding capabilities in its second capability set, and the first terminal can also receive the panoramic composite information transmitted through the first session channel. It is evident that the above embodiments of this application can realize panoramic video sessions between the first terminal and the second terminal, improving the user experience and meeting user session needs.

[0126] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the panoramic decoding capability, and the second expected media capability includes the front-facing camera-related capability and / or the rear-facing camera-related capability of the second terminal;

[0127] The first indication information is carried in the second update message, which is used to request the first terminal to update the first expected media capability to the front-facing camera-related capability and / or rear-facing camera-related capability of the first terminal.

[0128] Please see Figure 4 A media negotiation method in this application embodiment specifically includes the following steps:

[0129] b11. The first terminal initiates the first session, which carries the first expected media capability, including panoramic decoding capability.

[0130] It should be noted that, in the above embodiments, the first expected media capability indicates that the first terminal expects to conduct a panoramic video session with the second terminal, and the first terminal initiates a panoramic video call to the second terminal through the first session. At this time, the capability set of the first terminal includes the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first terminal supports panoramic video decoding and can output panoramic video.

[0131] b12. The second terminal receives the first session and sends the second expected media capability to the first terminal. The second expected media capability includes front camera-related capability and / or rear camera-related capability.

[0132] In the above steps, after the second terminal receives the first session, it can choose the session answering method according to the actual situation. For example, if the second terminal chooses the normal video method (front camera answering or rear camera answering) to answer the second session, then the second expected media capability of the second terminal is the front camera-related capability and / or the rear camera-related capability. It can be seen that at this time, the first and second expected media capabilities do not match, and media renegotiation needs to be performed through the server.

[0133] b13. The second terminal sends a second update message to the server;

[0134] b14. The server sends a second update message to the first terminal. The second update message is used to request the first terminal to update the first expected media capability to the first terminal's front-facing related capability and / or rear-facing related capability.

[0135] In practice, not only can the first terminal send a session error message to the server, informing it that the first media negotiation between the first and second terminals failed, but the second terminal can also proactively send a second update message to the server, informing it that the first media negotiation between the second and first terminals failed. Furthermore, the second terminal can also instruct the server to send an instruction to the first terminal via the second update message, instructing the first terminal to update its first expected media capability to its front-facing and / or rear-facing capabilities. In this way, the updated first and second expected media capabilities match, supporting a second session between the first and second terminals.

[0136] b15. The server releases dual-channel resources;

[0137] It's worth noting that after the server sends the second update message to the first terminal, it can also release dual video channel resources on the network side. During a video call, the network side releases network resources previously used to support video calls (such as bandwidth and connectivity) back to the system, allowing other users or applications to use these resources for communication or data transmission. This resource release operation helps optimize network resource utilization, ensuring network efficiency and performance.

[0138] b16. The server receives a second response message sent by the first terminal. The second response message carries the updated first expected media capability.

[0139] In the above steps, the first terminal, as the calling user, can send a second response message to the server. The second response message can be a status code of success (200 OK UPDATE), indicating that the request has been successfully received, understood and accepted.

[0140] b17. The server sends the seventh update message to the second terminal;

[0141] Specifically, after the first terminal completes the update of the first expected media capability, the server can send a seventh update message to the second terminal. The seventh update message instructs the second terminal to respond to the second session initiated by the first terminal based on the updated first expected media capability, at which point the updated first expected media capability matches the first expected media capability.

[0142] b18. The first terminal and the second terminal conduct a second session.

[0143] In a specific embodiment of this application, since the first expected media capability is panoramic decoding capability, and the second expected media capability is the front-facing camera-related capability and / or rear-facing camera-related capability of the second terminal, it is evident that when the first terminal initiates a panoramic video session request, but the second terminal expects to conduct a regular video call, their expected media capabilities do not match. In this case, the second terminal can send a second update message to the server, and the server will renegotiate the media with the first terminal, causing the first terminal to change its first expected media capability. The updated first and second expected media capabilities will then match, allowing the first and second terminals to conduct a second session. The first terminal can conduct the session via panoramic video, while the second terminal can conduct the session via regular video. This satisfies the session needs of each terminal user and optimizes the session process.

[0144] Optionally, initiating a second session with the second terminal using the first media negotiation method includes:

[0145] In the event that the dual-channel resources of the server are released, a second response message is sent to the server, the second response message carrying the updated first expected media capability;

[0146] Initiate the second session to the second terminal.

[0147] It is understandable that the aforementioned first negotiation method can be applied when the first expected media capability is panoramic decoding capability, but the second expected media capability is the front-facing camera-related capability and / or rear-facing camera-related capability of the second terminal, that is, when the first terminal expects to conduct a panoramic video session, and the second terminal expects to conduct a normal video session. In this case, the first terminal updates its first expected media capability to match the second expected media capability, releases dual-channel resources on the network side, and sends a second response message to the server, so that the server sends a seventh update message to the second terminal, realizing the second session between the first terminal and the second terminal.

[0148] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0149] The first indication information is carried in the third update message, which is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0150] like Figure 5 As shown, this application also provides a specific embodiment of a media negotiation method, which specifically includes the following steps:

[0151] c11. The first terminal initiates the first session, which carries the first expected media capability (SDP including front camera related capability (Video1-front) and / or rear camera related capability (Video2-bcck)).

[0152] It should be noted that when the first terminal expects to conduct a normal video session with the second terminal, the first expected media capability is the front-facing camera-related capability and / or the rear-facing camera-related capability, and the carried SDP includes Video1-front and / or Video2-bcck.

[0153] c12. The second terminal receives the first session initiated by the first terminal and sends the second expected media capability to the first terminal.

[0154] Furthermore, after receiving the first session, the second terminal can exchange expected media capabilities with the first terminal, sending a second expected media capability to the first terminal to inform it that it has selected panoramic video as the answering method. In addition, resource reservation can be performed during the exchange of expected media capabilities.

[0155] c13. The second terminal sends a third update message to the server, requesting media renegotiation with the first terminal;

[0156] c14. The server sends a third update message to the first terminal;

[0157] In the above steps, since the first expected media resource and the second expected media resource do not match, the second terminal can send a third update message to the server to request the first terminal to update the first expected media capability to the first terminal's panoramic decoding capability, so as to further realize media renegotiation with the first terminal.

[0158] c15. The first terminal receives the renegotiation request from the second terminal and executes the target operation;

[0159] In the specific steps, since the first terminal's first expected media capability is front-facing camera-related capability and / or rear-facing camera-related capability, while the second expected media capability is panoramic encoding capability and panoramic decoding capability, and since the first capability encompasses the first terminal's front-facing camera-related capability, rear-facing camera-related capability, and panoramic encoding capability, it is evident that the first terminal possesses the media capability for panoramic video conversations. To maximize the user's conversation experience, the second terminal can request the first terminal to change the first expected media capability, and actually request the first terminal to change the actual conversation mode to panoramic video mode. Therefore, to meet the user's needs of the second terminal, the first terminal can receive and agree to the second terminal's renegotiation request and execute the target operation. The renegotiation request is to change the first terminal's conversation initiation mode to panoramic video mode, and the target operation can be simultaneously opening the front-facing camera mode and the rear-facing camera mode, or opening the panoramic camera mode, etc.

[0160] c16. The first terminal sends a third response message to the server, the third response message carrying the updated first expected media capability;

[0161] In the above specific implementation steps, the first terminal can send a third response message to the server, which can carry the updated first expected media capabilities, such as panoramic decoding capabilities and / or panoramic encoding capabilities, to the media processing platform for updating.

[0162] c17. The server sends a third response message to the second terminal;

[0163] Furthermore, after the server receives the third response message, it can send a third response message to the second terminal to inform the second terminal that the first terminal has received the media renegotiation request.

[0164] c18. The fourth update message sent by the first terminal to the server, the fourth update message carrying the first capability set;

[0165] In addition, the first terminal can also send a fourth update message to the server, which can inform the server of its first set of capabilities and also inform the server that it can further conduct media negotiations with the second terminal.

[0166] c19. The server sends a fifth update message to the first terminal. The fifth update message is used to request the first terminal to establish a multi-channel media path.

[0167] Subsequently, the server can send a fifth update message to the first terminal. Upon receiving the fifth update message, the first terminal can establish a multi-channel media path. This multi-channel media includes front-facing camera-related media, rear-facing camera-related media, and panoramic-related media from the first terminal. The multi-channel media path can be used to transmit various multimedia resources between the server and the first terminal.

[0168] c20. When the first terminal completes the establishment of the multi-channel media path, the server receives the fourth response message sent by the first terminal and establishes a second session channel with the first terminal based on the multi-channel media path.

[0169] Specifically, based on the fourth response message, the server determines that the first terminal has completed the establishment of the multi-channel media path. At this point, a second session channel can be established between the first terminal and the server based on the multi-channel media path. This channel is used by the server to collect the multimedia resources of the first terminal and transmit the synthesized media resources from the server to the first terminal.

[0170] c21. Based on the second session channel, the server synthesizes panoramic composite information;

[0171] Specifically, since the first capability set lacks panoramic encoding capabilities, the first terminal cannot acquire global video, global image, or other media information. Therefore, global composite information can be obtained through server synthesis. The server can acquire at least one piece of second information collected by the first terminal based on various media capabilities within its first capability set via a second session channel. Subsequently, multiple pieces of second information are synthesized to obtain panoramic composite information, which is then transmitted to the first terminal via the second session channel.

[0172] c22. The second terminal sends the eighth update message to the server;

[0173] Understandably, after the establishment of the second session channel is completed, the second terminal can also send an eighth update message to the server. The eighth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal.

[0174] b23. The first terminal and the second terminal conduct a second session.

[0175] In the above embodiments of this application, the first terminal desires a regular video session, while the second terminal desires a global video session. Since the first terminal possesses global receiving capabilities, a two-way video session can be achieved. To maximize the session effect, the second terminal requests the first terminal to update its desired media capabilities through the server and performs the target operation. Additionally, panoramic synthesis information can be transmitted to the first terminal through the second session channel, addressing the issue of the first terminal lacking panoramic encoding capabilities. Thus, both the first and second terminals can conduct panoramic video sessions, maximizing the fulfillment of user session needs, optimizing session services, and improving user experience.

[0176] Optionally, initiating a second session with the second terminal using the first media negotiation method includes:

[0177] Send a third response message to the server, the third response message carrying the updated first expected media capability;

[0178] Send a fourth update message to the server, the fourth update message carrying the first capability set;

[0179] The server sends a fifth update message, which requests the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0180] When the first terminal establishes the multi-channel media path, a fourth response message is sent to the server, and a second session channel based on the multi-channel media path is established with the server;

[0181] Initiate a second session to the second terminal.

[0182] In some embodiments of this application, the first media negotiation method can also be applied when the first expected media capability is a front-facing camera-related capability and / or a rear-facing camera-related capability, and the second expected media capability is a panoramic encoding capability and a panoramic decoding capability. In this case, to meet the session needs of the end user, the second terminal can request the first terminal to update the first expected media capability through the server. Further, the server can send a first instruction message to the first terminal and a second instruction message to the second terminal. The first instruction message instructs the first terminal to adopt the first media negotiation method, establish a second session channel with the server, and obtain the panoramic composite information synthesized by the server to compensate for the lack of panoramic encoding capability. The second instruction message instructs the second terminal to adopt the second media negotiation method, that is, to respond to the second session of the first terminal and realize a two-way panoramic video session with the first terminal.

[0183] Optionally, before initiating the first session to the second terminal, the method further includes:

[0184] Send the first capability set to the server.

[0185] In some other embodiments, before the first terminal initiates a first session to the second terminal, and before the second terminal receives the first session from the first terminal, the first terminal and the second terminal may respectively send their respective capability sets to the server, namely a first capability set and a second capability set. This allows the first terminal and the second terminal to register their media capabilities with the server before the session is established. This facilitates the server's orchestration of the registered media capabilities to determine the media capabilities to be used during the rewrite negotiation between the two parties, updating the session to ensure the continuity of the session process and improve the adaptability of the sessions between the two parties.

[0186] Please refer to Figure 5This application also provides a media negotiation method applied to a server, specifically including the following steps:

[0187] Step 301: Receive first media capability information and second media capability information sent by the first terminal and the second terminal respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability.

[0188] Step 402: Determine the first media negotiation method based on the second expected media capability, and determine the second media negotiation method based on the first expected media capability information;

[0189] Step 503: Send a first instruction message to the first terminal, the first instruction message being used to indicate the first media negotiation method; and send a second instruction message to the second terminal, the second instruction message being used to indicate the second media negotiation method.

[0190] Optionally, the first media capability information includes a first capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal; the first expected media capability includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal.

[0191] The second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic-related capabilities of the second terminal; the second expected media capability includes at least one of the panoramic encoding capabilities and panoramic decoding capabilities of the second terminal.

[0192] Optionally, when the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, sending the first indication information to the first terminal includes:

[0193] Send a first update message to the first terminal, the first indication information is carried in the first update message, the first update message is used to request the first terminal to establish a multi-channel media path, the multi-channel media path includes a media path established based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal;

[0194] When the first terminal establishes the multi-channel media path, the system receives the first response message sent by the first terminal and establishes a first session channel with the first terminal based on the multi-channel media path.

[0195] Optionally, sending the second indication information to the second terminal includes:

[0196] A sixth update message is sent to the second terminal, wherein the second indication information is carried in the sixth update message, and the sixth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the first session channel;

[0197] Based on the first session channel, panoramic composite information is sent to the first terminal.

[0198] Optionally, sending panoramic composite information to the first terminal based on the first session channel includes:

[0199] Based on the first session channel, at least one piece of first information is obtained by the first terminal based on the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal;

[0200] By synthesizing the at least one first piece of information, the panoramic composite information is obtained;

[0201] The panoramic synthesis information is sent to the second terminal.

[0202] Optionally, when the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, sending the first indication information to the first terminal includes:

[0203] Receive a second update message sent by the second terminal, the second update message being used to request the first terminal to update the first expected media capability to the first terminal's front-facing camera-related capability and / or rear-facing camera-related capability;

[0204] Send the second update message to the first terminal and release the dual-channel resources, with the first indication information carried in the second update message;

[0205] Upon completion of the dual-channel resource release, a second response message is received from the first terminal, the second response message carrying the updated first expected media capability.

[0206] Optionally, sending the second indication information to the second terminal includes:

[0207] A seventh update message is sent to the second terminal, in which the second indication information is carried. The seventh update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the updated first expected media capability.

[0208] Optionally, when the first capability set includes the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic coding capabilities, sending the first indication information to the first terminal includes:

[0209] The system receives a third update message sent by the second terminal, wherein the first indication information is carried in the third update message; wherein the third update message is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0210] Send the third update message to the first terminal;

[0211] The system receives a third response message sent by the first terminal and sends the third response message to the second terminal. The third response message carries the updated first expected media capability.

[0212] Receive a fourth update message sent by the first terminal, the fourth update message carrying the first capability set;

[0213] A fifth update message is sent to the first terminal, the fifth update message being used to request the first terminal to establish multiple media paths, the multiple media paths including media paths based on the first terminal's front-facing camera correlation capabilities and rear-facing camera correlation capabilities;

[0214] When the first terminal completes the establishment of the multi-channel media path, it receives the fourth response message sent by the first terminal and establishes a second session channel with the first terminal based on the multi-channel media path.

[0215] Optionally, sending the second indication information to the second terminal includes:

[0216] Send an eighth update message to the second terminal, wherein the second indication information is carried in the eighth update message, and the eighth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal when the multi-channel media path is established;

[0217] Panoramic composite information is sent to the first terminal based on the second session channel.

[0218] Based on the above, this application provides a media negotiation method for servers to solve the problem of session failures caused by different media capabilities in related technologies. This application can reduce the probability of session failures between the first terminal and the second terminal due to differences in media capabilities when the first terminal and the second terminal use different media capabilities, thereby optimizing session services and improving the user's communication session experience. The media negotiation method in the above embodiments is similar to the aforementioned method... Figure 1 and Figure 7 The media negotiation method described in the previous application is based on the same concept. Since the principles of solving the problem are similar, the implementation of the media negotiation method described in this application can be referred to the embodiments shown above. Repeated parts will not be described again.

[0219] like Figure 7 As shown, this application also provides a media negotiation method applied to a second terminal, the method specifically including:

[0220] Step 401: Receive a first session initiated by a first terminal. The first session carries first media capability information, which includes a first expected media capability.

[0221] Step 402: Send the second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal;

[0222] Step 403: If the first expected media capability and the second expected media capability do not match, report the first expected media capability to the server;

[0223] Step 404: Receive the second indication information sent by the server. The second indication information is used to indicate a second media negotiation method, which is determined based on the first expected media capability.

[0224] Step 405: Respond to the second session initiated by the first terminal using the second media negotiation method.

[0225] Optionally, the second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic capabilities of the second terminal.

[0226] The second expected media capability includes at least one of the second terminal's panoramic encoding capability and panoramic decoding capability.

[0227] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first expected media capability is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0228] The receipt of the second indication information sent by the server includes:

[0229] Receive the sixth update message sent by the server, wherein the second indication information is carried in the sixth update message;

[0230] The sixth update message is used to respond to the second session initiated by the first terminal based on the first session channel when a first session channel is established between the first terminal and the server.

[0231] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0232] The receipt of the second indication information sent by the server includes:

[0233] Send a second update message to the server, the second update message being used to request the first terminal to update the first expected media capability to the first terminal's front-facing camera-related capability and / or rear-facing camera-related capability;

[0234] When the server completes the dual-channel resource release, a seventh update message is received from the server, and the second indication information is carried in the seventh update message;

[0235] The seventh update message is used to respond to the second session initiated by the first terminal based on the updated first expected media capability, wherein the updated first expected media capability is the front camera related capability and the rear camera related capability of the first terminal.

[0236] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic encoding capability of the first terminal, the first expected media capability in the first session is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0237] The receipt of the second indication information sent by the server includes:

[0238] Send a third update message to the server, the third update message being used to request the first terminal to update the first expected media capability to the first terminal's panoramic decoding capability;

[0239] The first terminal receives a third response message sent by the server. The third response message is transmitted from the first terminal to the server and carries the updated first expected media capability, which is the panoramic decoding capability of the first terminal.

[0240] When the first terminal and the server complete the establishment of the second session channel, the server sends an eighth update message, in which the second indication information is carried; wherein the eighth update message is used to respond to the second session initiated by the first terminal.

[0241] Optionally, before receiving the first session initiated by the first terminal, the method further includes:

[0242] Send the second capability set to the server.

[0243] Based on the above, this application provides a media negotiation method for a second terminal to solve the problem of session failure easily caused by different media capabilities in related technologies. The media negotiation method in the above embodiments is similar to the one described above. Figure 1 and Figure 6 The media negotiation method described in the previous application is based on the same concept. Since the principles of solving the problem are similar, the implementation of the media negotiation method described in this application can be referred to the embodiments shown above. Repeated parts will not be described again.

[0244] This application also provides a media negotiation device applied to a first terminal, see [link to relevant documentation]. Figure 8 , Figure 8 This is one of the structural schematic diagrams of the media negotiation device provided in the embodiments of this application. Since the principle of the media negotiation device in solving the problem is similar to that of the media negotiation method in the embodiments of this application, the implementation of this media negotiation device can refer to the implementation of this method, and the repeated parts will not be described again.

[0245] like Figure 8 As shown, the media negotiation device applied to the first terminal includes:

[0246] The first session module 511 is used to initiate a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability.

[0247] The first reporting module 512 is used to report the second expected media capability to the server when the first expected media capability does not match the second expected media capability sent by the second terminal.

[0248] The first receiving module 513 is configured to receive first indication information sent by the server, the first indication information being used to indicate a first media negotiation mode, the first media negotiation mode being determined based on the second expected media capability;

[0249] The second session module 514 is used to initiate a second session to the second terminal using the first media negotiation method.

[0250] Optionally, the first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities;

[0251] The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

[0252] Optionally, when the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0253] The first reporting module 512 includes:

[0254] The first sending unit is configured to send a session error message to the server, the session error message including the second expected media capability, and the session error message is used to indicate that the negotiation between the first terminal and the second terminal failed;

[0255] The first indication information is carried in the first update message, which is used to request the first terminal to establish multiple media paths. The multiple media paths include media paths based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0256] Optionally, the second session module 514 includes:

[0257] The second sending unit is used to send a first response message to the server when the first terminal establishes the multi-channel media path, and to establish a first session channel with the server based on the multi-channel media path.

[0258] The first session unit is used to initiate a second session to the second terminal.

[0259] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the panoramic decoding capability, and the second expected media capability includes the front-facing camera-related capability and / or the rear-facing camera-related capability of the second terminal;

[0260] The first indication information is carried in the second update message, which is used to request the first terminal to update the first expected media capability to the front-facing camera-related capability and / or rear-facing camera-related capability of the first terminal.

[0261] Optionally, the second session module 514 includes:

[0262] The third sending unit is used to send a second response message to the server when the dual-channel resources of the server are released, the second response message carrying the updated first expected media capability;

[0263] The second session unit is used to initiate the second session to the second terminal.

[0264] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0265] The first indication information is carried in the third update message, which is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0266] Optionally, the second session module 514 includes:

[0267] The fourth sending unit is used to send a third response message to the server, the third response message carrying the updated first expected media capability;

[0268] The fifth sending unit is used to send a fourth update message to the server, the fourth update message carrying the first capability set;

[0269] The first receiving unit is configured to receive a fifth update message sent by the server. The fifth update message is configured to request the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0270] The sixth sending unit is used to send a fourth response message to the server when the first terminal establishes the multi-channel media path, and to establish a second session channel with the server based on the multi-channel media path.

[0271] The third session unit is used to initiate a second session to the second terminal.

[0272] Optionally, before initiating the first session to the second terminal, the media negotiation device further includes:

[0273] The third sending module is used to send the first capability set to the server.

[0274] The media negotiation device provided in this application embodiment can perform the above-described... Figure 1 The media negotiation method embodiment shown is similar in principle and technical effect, and will not be described again here.

[0275] This application also provides a media negotiation device for a server. The principle behind the problem-solving approach of the media negotiation device is similar to that in this application. Figure 6 The media negotiation method shown is similar; therefore, the implementation of this media negotiation device can be found in the implementation of the method, and repeated details will not be elaborated further. For example... Figure 9 As shown in the embodiments of this application, the media negotiation device applied to the server includes:

[0276] The second receiving module 521 is used to receive first media capability information and second media capability information sent by the first terminal and the second terminal respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability.

[0277] The negotiation determination module 522 is used to determine a first media negotiation method based on the second expected media capability, and to determine a second media negotiation method based on the first expected media capability information;

[0278] The first sending module 523 is configured to send first indication information to the first terminal, the first indication information being used to indicate the first media negotiation mode, and to send second indication information to the second terminal, the second indication information being used to indicate the second media negotiation mode.

[0279] Optionally, the first media capability information includes a first capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal; the first expected media capability includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal.

[0280] The second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic-related capabilities of the second terminal; the second expected media capability includes at least one of the panoramic encoding capabilities and panoramic decoding capabilities of the second terminal.

[0281] Optionally, when the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first sending module 523 includes:

[0282] The seventh sending unit is used to send a first update message to the first terminal. The first indication information is carried in the first update message. The first update message is used to request the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path established based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0283] The second receiving unit is configured to receive a first response message sent by the first terminal when the first terminal establishes the multi-channel media path, and to establish a first session channel with the first terminal based on the multi-channel media path.

[0284] Optionally, the first transmitting module 523 includes:

[0285] The eighth sending unit is used to send a sixth update message to the second terminal, wherein the second indication information is carried in the sixth update message, and the sixth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the first session channel;

[0286] The ninth sending unit is used to send panoramic composite information to the first terminal based on the first session channel.

[0287] Optionally, the ninth transmitting unit includes:

[0288] The acquisition subunit is used to acquire at least one piece of first information collected by the first terminal based on the first session channel and the first terminal’s front-facing related capabilities and rear-facing related capabilities.

[0289] A synthesis subunit is used to synthesize the at least one first piece of information to obtain the panoramic synthesis information;

[0290] The transmitting subunit is used to send the panoramic synthesis information to the second terminal.

[0291] Optionally, when the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first sending module 523 includes:

[0292] The third receiving unit is configured to receive a second update message sent by the second terminal, wherein the second update message is configured to request the first terminal to update the first expected media capability to the first terminal’s front-facing related capability and / or rear-facing related capability.

[0293] The tenth sending unit is used to send the second update message to the first terminal and release dual-channel resources, wherein the first indication information is carried in the second update message;

[0294] The fourth receiving unit is configured to receive a second response message sent by the first terminal after the dual-channel resource release is completed. The second response message carries the updated first expected media capability.

[0295] Optionally, the first transmitting module 523 includes:

[0296] The eleventh sending unit is used to send a seventh update message to the second terminal, wherein the second indication information is carried in the seventh update message, and the seventh update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the updated first expected media capability.

[0297] Optionally, when the first capability set includes the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic encoding capabilities of the first terminal, the first sending module 523 includes:

[0298] The fifth receiving unit is configured to receive a third update message sent by the second terminal, wherein the first indication information is carried in the third update message; wherein the third update message is configured to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0299] The twelfth sending unit is used to send the third update message to the first terminal;

[0300] The sixth receiving unit is configured to receive the third response message sent by the first terminal and send the third response message to the second terminal, wherein the third response message carries the updated first expected media capability;

[0301] The seventh receiving unit is used to receive the fourth update message sent by the first terminal, wherein the fourth update message carries the first capability set;

[0302] The thirteenth sending unit is used to send a fifth update message to the first terminal. The fifth update message is used to request the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0303] The eighth receiving unit is configured to receive a fourth response message sent by the first terminal when the first terminal completes the establishment of the multi-channel media path, and to establish a second session channel with the first terminal based on the multi-channel media path.

[0304] Optionally, the first transmitting module 523 includes:

[0305] The fourteenth sending unit is used to send an eighth update message to the second terminal, wherein the second indication information is carried in the eighth update message, and the eighth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal when the multi-channel media path is established;

[0306] The fifteenth sending unit is used to send panoramic composite information to the first terminal based on the second session channel.

[0307] Please refer to Figure 10 , Figure 10 This is the third schematic diagram of a media negotiation device in the embodiments of this application, applied to a second terminal. The device specifically includes:

[0308] The third receiving module 531 is used to receive a first session initiated by the first terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability;

[0309] The second sending module 532 is used to send a second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal;

[0310] The second reporting module 533 is used to report the first expected media capability to the server when the first expected media capability and the second expected media capability do not match.

[0311] The fourth receiving module 534 is used to receive the second indication information sent by the server, the second indication information being used to indicate a second media negotiation mode, the second media negotiation mode being determined based on the first expected media capability;

[0312] The session response module 535 is used to respond to the second session initiated by the first terminal using the second media negotiation method.

[0313] Optionally, the second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic capabilities of the second terminal.

[0314] The second expected media capability includes at least one of the second terminal's panoramic encoding capability and panoramic decoding capability.

[0315] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first expected media capability is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0316] The fourth receiving module 534 includes:

[0317] The ninth receiving unit is used to receive the sixth update message sent by the server, wherein the second indication information is carried in the sixth update message;

[0318] The sixth update message is used to respond to the second session initiated by the first terminal based on the first session channel when a first session channel is established between the first terminal and the server.

[0319] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0320] The fourth receiving module 534 includes:

[0321] The sixteenth sending unit is used to send a second update message to the server. The second update message is used to request the first terminal to update the first expected media capability to the first terminal's front-facing related capability and / or rear-facing related capability.

[0322] The tenth receiving unit is configured to receive a seventh update message sent by the server when the server completes the dual-channel resource release, wherein the second indication information is carried in the seventh update message;

[0323] The seventh update message is used to respond to the second session initiated by the first terminal based on the updated first expected media capability, wherein the updated first expected media capability is the front camera related capability and the rear camera related capability of the first terminal.

[0324] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic encoding capability of the first terminal, the first expected media capability in the first session is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0325] The fourth receiving module 534 includes:

[0326] The seventeenth sending unit is used to send a third update message to the server, the third update message being used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal;

[0327] The eleventh receiving unit is used to receive a third response message sent by the server. The third response message is transmitted from the first terminal to the server and carries the updated first expected media capability, which is the panoramic decoding capability of the first terminal.

[0328] The twelfth receiving unit is configured to receive an eighth update message sent by the server when the first terminal and the server have completed the establishment of a second session channel, wherein the second indication information is carried in the eighth update message; wherein the eighth update message is used to respond to the second session initiated by the first terminal.

[0329] Optionally, prior to receiving the first session initiated by the first terminal, the media negotiation device further includes:

[0330] The fourth sending module is used to send the second capability set to the server.

[0331] It should be noted that the aforementioned media negotiation device is capable of achieving... Figure 7 The various processes implemented in the media negotiation method embodiment shown herein can achieve the same beneficial effects, and will not be described again here to avoid repetition.

[0332] like Figure 11 As shown, the communication device in this embodiment includes a processor 600, configured to read a program from a memory 620 and execute the following processes:

[0333] The transceiver 610 initiates a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability.

[0334] If the first expected media capability does not match the second expected media capability sent by the second terminal, the second expected media capability is reported to the server through the transceiver 610.

[0335] The transceiver 610 receives first indication information sent by the server, the first indication information being used to indicate a first media negotiation mode, the first media negotiation mode being determined based on the second expected media capability;

[0336] The second session is initiated to the second terminal via transceiver 610 using the first media negotiation method;

[0337] Transceiver 610 is used to receive and send data under the control of processor 600.

[0338] Among them, Figure 11In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 600 and memory represented by memory 620 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 610 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 630 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0339] The processor 600 is responsible for managing the bus architecture and general processing, while the memory 620 can store the data used by the processor 600 when performing operations.

[0340] Optionally, the first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities;

[0341] The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

[0342] Optionally, when the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0343] Processor 600 is used to read the program from memory 620 and execute the following procedures:

[0344] The transceiver 610 sends a session error message to the server, the session error message including the second expected media capability, and the session error message is used to indicate that the negotiation between the first terminal and the second terminal has failed;

[0345] The first indication information is carried in the first update message, which is used to request the first terminal to establish multiple media paths. The multiple media paths include media paths based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0346] Optionally, the processor 600 is configured to read the program from the memory 620 and execute the following procedures:

[0347] When the first terminal establishes the multi-media path, the transceiver 610 sends a first response message to the server and establishes a first session channel with the server based on the multi-media path.

[0348] A second session is initiated to the second terminal via transceiver 610;

[0349] Based on the first session channel, panoramic information is acquired in the second session and transmitted to the second terminal via transceiver 610.

[0350] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the panoramic decoding capability, and the second expected media capability includes the front-facing camera-related capability and / or the rear-facing camera-related capability of the second terminal;

[0351] The first indication information is carried in the second update message, which is used to request the first terminal to update the first expected media capability to the front-facing camera-related capability and / or rear-facing camera-related capability of the first terminal.

[0352] Optionally, the processor 600 is configured to read the program from the memory 620 and execute the following procedures:

[0353] In the event that the dual-channel resources of the server are released, a second response message is sent to the server via transceiver 610, the second response message carrying the updated first expected media capability;

[0354] The second session is initiated to the second terminal via transceiver 610.

[0355] Optionally, when the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0356] The first indication information is carried in the third update message, which is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0357] Optionally, the processor 600 is configured to read the program from the memory 620 and execute the following procedures:

[0358] A third response message is sent to the server via transceiver 610, the third response message carrying the updated first expected media capability;

[0359] A fourth update message is sent to the server via transceiver 610, the fourth update message carrying the first capability set;

[0360] The transceiver 610 receives the fifth update message sent by the server. The fifth update message is used to request the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0361] When the first terminal establishes the multi-channel media path, the transceiver 610 sends a fourth response message to the server and establishes a second session channel with the server based on the multi-channel media path.

[0362] A second session is initiated to the second terminal via transceiver 610;

[0363] Based on the second session channel, panoramic information is acquired in the second session and transmitted to the second terminal through transceiver 610.

[0364] Optionally, the processor 600 is configured to read the program from the memory 620 and execute the following procedures:

[0365] The first capability set is sent to the server via transceiver 610.

[0366] The communication device provided in this application embodiment can perform the above-described... Figure 1 The media negotiation method embodiment shown is similar in principle and technical effect, and will not be described again here.

[0367] like Figure 12 As shown, the communication device in this embodiment includes a processor 700, configured to read a program from a memory 720 and execute the following processes:

[0368] The transceiver 710 receives first media capability information and second media capability information sent by the first terminal and the second terminal respectively. The first media capability information includes a first expected media capability and the second media capability information includes a second expected media capability.

[0369] The first media negotiation method is determined based on the second expected media capability, and the second media negotiation method is determined based on the first expected media capability information;

[0370] The transceiver 710 sends a first instruction message to the first terminal, the first instruction message being used to indicate the first media negotiation mode, and sends a second instruction message to the second terminal, the second instruction message being used to indicate the second media negotiation mode.

[0371] Transceiver 710 is used to receive and send data under the control of processor 700.

[0372] Among them, Figure 12 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 700 and memory represented by memory 720 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 710 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 730 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0373] The processor 700 is responsible for managing the bus architecture and general processing, while the memory 720 can store the data used by the processor 700 during operation.

[0374] Optionally, the first media capability information includes a first capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal; the first expected media capability includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities of the first terminal.

[0375] The second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic-related capabilities of the second terminal; the second expected media capability includes at least one of the panoramic encoding capabilities and panoramic decoding capabilities of the second terminal.

[0376] Optionally, if the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the processor 700 is configured to read the program in the memory 720 and execute the following process:

[0377] The transceiver 710 sends a first update message to the first terminal, the first indication information is carried in the first update message, the first update message is used to request the first terminal to establish a multi-channel media path, the multi-channel media path includes a media path established based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal;

[0378] When the first terminal establishes the multi-channel media path, the transceiver 710 receives the first response message sent by the first terminal and establishes a first session channel with the first terminal based on the multi-channel media path.

[0379] Optionally, the processor 700 is configured to read the program from the memory 720 and execute the following procedures:

[0380] The transceiver 710 sends a sixth update message to the second terminal, the second indication information being carried in the sixth update message, the sixth update message being used to instruct the second terminal to respond to the second session initiated by the first terminal based on the first session channel;

[0381] Based on the first session channel, panoramic composite information is sent to the first terminal via transceiver 710.

[0382] Optionally, the processor 700 is configured to read the program from the memory 720 and execute the following procedures:

[0383] Based on the first session channel, at least one piece of first information is obtained by the first terminal based on the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal;

[0384] By synthesizing the at least one first piece of information, the panoramic composite information is obtained;

[0385] The panoramic synthesis information is sent to the second terminal via transceiver 710.

[0386] Optionally, if the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the processor 700 is configured to read the program in the memory 720 and execute the following process:

[0387] The transceiver 710 receives a second update message sent by the second terminal. The second update message is used to request the first terminal to update the first expected media capability to the first terminal's front-facing camera-related capability and / or rear-facing camera-related capability.

[0388] The transceiver 710 sends the second update message to the first terminal and releases the dual-channel resources, with the first indication information carried in the second update message;

[0389] Upon completion of the dual-channel resource release, the transceiver 710 receives a second response message sent by the first terminal, the second response message carrying the updated first expected media capability.

[0390] Optionally, the processor 700 is configured to read the program from the memory 720 and execute the following procedures:

[0391] A seventh update message is sent to the second terminal via transceiver 710. The second indication information is carried in the seventh update message. The seventh update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the updated first expected media capability.

[0392] Optionally, when the first capability set includes the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic encoding capabilities of the first terminal, the processor 700 is configured to read the program from the memory 720 and execute the following process:

[0393] The transceiver 710 receives a third update message sent by the second terminal, in which the first indication information is carried; wherein, the third update message is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

[0394] The third update message is sent to the first terminal via transceiver 710;

[0395] The transceiver 710 receives the third response message sent by the first terminal and sends the third response message to the second terminal. The third response message carries the updated first expected media capability.

[0396] The transceiver 710 receives a fourth update message sent by the first terminal, the fourth update message carrying the first capability set;

[0397] The transceiver 710 sends a fifth update message to the first terminal. The fifth update message is used to request the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

[0398] When the first terminal completes the establishment of the multi-channel media path, the transceiver 710 receives the fourth response message sent by the first terminal and establishes a second session channel with the first terminal based on the multi-channel media path.

[0399] Optionally, the processor 700 is configured to read the program from the memory 720 and execute the following procedures:

[0400] The transceiver 710 sends an eighth update message to the second terminal, the second indication information being carried in the eighth update message. The eighth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal when the multi-channel media path is established.

[0401] Based on the second session channel, panoramic composite information is sent to the first terminal via transceiver 710.

[0402] The communication device provided in this application embodiment can perform the above-described... Figure 6 The media negotiation method embodiment shown is similar in principle and technical effect, and will not be described again here.

[0403] like Figure 13 As shown, the communication device in this embodiment includes a processor 800, configured to read a program from a memory 820 and execute the following processes:

[0404] The transceiver 810 receives a first session initiated by a first terminal. The first session carries first media capability information, which includes a first expected media capability.

[0405] The transceiver 810 sends a second expected media capability to the first terminal, the second expected media capability being the expected media capability of the second terminal;

[0406] If the first expected media capability and the second expected media capability do not match, the first expected media capability is reported to the server through transceiver 810;

[0407] The transceiver 810 receives a second indication information sent by the server, the second indication information being used to indicate a second media negotiation mode, the second media negotiation mode being determined based on the first expected media capability;

[0408] The second session initiated by the first terminal is responded to using the second media negotiation method;

[0409] Transceiver 610 is used to receive and send data under the control of processor 800.

[0410] Among them, Figure 13In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 800 and memory represented by memory 820 together. The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 810 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 830 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0411] The processor 800 is responsible for managing the bus architecture and general processing, while the memory 820 can store the data used by the processor 800 during operation.

[0412] Optionally, the second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic capabilities of the second terminal.

[0413] The second expected media capability includes at least one of the second terminal's panoramic encoding capability and panoramic decoding capability.

[0414] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first expected media capability is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0415] Processor 800 is also used to read from memory 820 and perform the following steps:

[0416] The transceiver 810 receives the sixth update message sent by the server, in which the second indication information is carried.

[0417] The sixth update message is used to respond to the second session initiated by the first terminal based on the first session channel when a first session channel is established between the first terminal and the server.

[0418] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0419] Processor 800 is also used to read from memory 820 and perform the following steps:

[0420] The transceiver 810 sends a second update message to the server, the second update message being used to request the first terminal to update the first expected media capability to the first terminal's front-facing related capability and / or rear-facing related capability;

[0421] When the server completes the dual-channel resource release, the transceiver 810 receives the seventh update message sent by the server, and the second indication information is carried in the seventh update message;

[0422] The seventh update message is used to respond to the second session initiated by the first terminal based on the updated first expected media capability, wherein the updated first expected media capability is the front camera related capability and the rear camera related capability of the first terminal.

[0423] Optionally, the first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic encoding capability of the first terminal, the first expected media capability in the first session is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal.

[0424] Processor 800 is also used to read from memory 820 and perform the following steps:

[0425] The transceiver 810 sends a third update message to the server, the third update message being used to request the first terminal to update the first expected media capability to the first terminal's panoramic decoding capability.

[0426] The transceiver 810 receives a third response message sent by the server. The third response message is transmitted from the first terminal to the server and carries the updated first expected media capability, which is the panoramic decoding capability of the first terminal.

[0427] When the first terminal and the server complete the establishment of the second session channel, the transceiver 810 receives the eighth update message sent by the server, in which the second indication information is carried; wherein, the eighth update message is used to respond to the second session initiated by the first terminal.

[0428] Optionally, before receiving the first session initiated by the first terminal, the processor 800 is also configured to read from the memory 820 and perform the following steps:

[0429] The second capability set is sent to the server via transceiver 810.

[0430] The communication device provided in this application embodiment can perform the above-described... Figure 7 The media negotiation method embodiment shown is similar in principle and technical effect, and will not be described again here.

[0431] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described functionality. Figure 1 , Figure 6 and Figure 7 The various processes of the illustrated method embodiments achieve the same technical effect, and will not be described again here to avoid repetition. The computer-readable storage medium mentioned includes, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0432] This application embodiment also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the above. Figure 1 , Figure 6 and Figure 7 The various processes in the method embodiments described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0433] This application embodiment also provides a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the above. Figure 1 , Figure 6 and Figure 7 The various processes in the method embodiments described herein can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0434] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0435] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0436] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the transmission and reception methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0437] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A media negotiation method, characterized in that, Applied to a first terminal, the method includes: A first session is initiated to the second terminal. The first session carries first media capability information, which includes a first expected media capability. If the first expected media capability does not match the second expected media capability sent by the second terminal, the second expected media capability shall be reported to the server. Receive first indication information sent by the server, the first indication information being used to indicate a first media negotiation method, the first media negotiation method being determined based on the second expected media capability; The second session is initiated to the second terminal using the first media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

2. The method according to claim 1, characterized in that, When the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal. The step of reporting the second expected media capability to the server includes: Send a session error message to the server, the session error message including the second expected media capability, and the session error message being used to indicate that the negotiation between the first terminal and the second terminal failed; The first indication information is carried in the first update message, which is used to request the first terminal to establish multiple media paths. The multiple media paths include media paths based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal.

3. The method according to claim 2, characterized in that, The step of initiating a second session with the second terminal using the first media negotiation method includes: When the first terminal establishes the multi-channel media path, a first response message is sent to the server, and a first session channel based on the multi-channel media path is established with the server; Initiate a second session to the second terminal.

4. The method according to claim 1, characterized in that, When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the panoramic decoding capability, and the second expected media capability includes the front-facing camera-related capability and the rear-facing camera-related capability of the second terminal; The first indication information is carried in the second update message, which is used to request the first terminal to update the first expected media capability to the front-facing camera-related capability and / or rear-facing camera-related capability of the first terminal.

5. The method according to claim 4, characterized in that, The step of initiating a second session with the second terminal using the first media negotiation method includes: In the event that the dual-channel resources of the server are released, a second response message is sent to the server, the second response message carrying the updated first expected media capability; Initiate the second session to the second terminal.

6. The method according to claim 1, characterized in that, When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic decoding capability of the first terminal, the first expected media capability is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal. The first indication information is carried in the third update message, which is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal.

7. The method according to claim 6, characterized in that, The step of initiating a second session with the second terminal using the first media negotiation method includes: Send a third response message to the server, the third response message carrying the updated first expected media capability; Send a fourth update message to the server, the fourth update message carrying the first capability set; The server sends a fifth update message, which requests the first terminal to establish a multi-channel media path. The multi-channel media path includes a media path based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal. When the first terminal establishes the multi-channel media path, a fourth response message is sent to the server, and a second session channel based on the multi-channel media path is established with the server; Initiate a second session to the second terminal.

8. The method according to any one of claims 1 to 7, characterized in that, Before initiating the first session to the second terminal, the method further includes: Send the first capability set to the server.

9. A media negotiation method, characterized in that, Applied to a server, the method includes: The system receives first media capability information and second media capability information sent by a first terminal and a second terminal, respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability. The first media negotiation method is determined based on the second expected media capability, and the second media negotiation method is determined based on the first expected media capability information; Send a first indication message to the first terminal, the first indication message being used to indicate the first media negotiation method; and send a second indication message to the second terminal, the second indication message being used to indicate the second media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

10. The method according to claim 9, characterized in that, The second media capability information includes a second capability set, which includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic-related capabilities of the second terminal; the second expected media capability includes at least one of the panoramic encoding capabilities and panoramic decoding capabilities of the second terminal.

11. The method according to claim 10, characterized in that, When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, sending the first indication information to the first terminal includes: Send a first update message to the first terminal, the first indication information is carried in the first update message, the first update message is used to request the first terminal to establish a multi-channel media path, the multi-channel media path includes a media path established based on the front-facing camera correlation capability and the rear-facing camera correlation capability of the first terminal; When the first terminal establishes the multi-channel media path, the system receives the first response message sent by the first terminal and establishes a first session channel with the first terminal based on the multi-channel media path.

12. The method according to claim 11, characterized in that, Sending the second instruction information to the second terminal includes: A sixth update message is sent to the second terminal, wherein the second indication information is carried in the sixth update message, and the sixth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the first session channel; Based on the first session channel, panoramic composite information is sent to the first terminal.

13. The method according to claim 12, characterized in that, Sending panoramic composite information to the first terminal based on the first session channel includes: Based on the first session channel, at least one piece of first information is obtained by the first terminal based on the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal; By synthesizing the at least one first piece of information, the panoramic composite information is obtained; The panoramic synthesis information is sent to the second terminal.

14. The method according to claim 10, characterized in that, When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, sending the first indication information to the first terminal includes: Receive a second update message sent by the second terminal, the second update message being used to request the first terminal to update the first expected media capability to the first terminal's front-facing camera-related capability and / or rear-facing camera-related capability; Send the second update message to the first terminal and release the dual-channel resources, with the first indication information carried in the second update message; Upon completion of the dual-channel resource release, a second response message is received from the first terminal, the second response message carrying the updated first expected media capability.

15. The method according to claim 13, characterized in that, Sending the second instruction information to the second terminal includes: A seventh update message is sent to the second terminal, in which the second indication information is carried. The seventh update message is used to instruct the second terminal to respond to the second session initiated by the first terminal based on the updated first expected media capability.

16. The method according to claim 10, characterized in that, When the first capability set includes the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic encoding capabilities, sending the first indication information to the first terminal includes: The system receives a third update message sent by the second terminal, wherein the first indication information is carried in the third update message; wherein the third update message is used to request the first terminal to update the first expected media capability to the panoramic decoding capability of the first terminal. Send the third update message to the first terminal; The system receives a third response message sent by the first terminal and sends the third response message to the second terminal. The third response message carries the updated first expected media capability. Receive a fourth update message sent by the first terminal, the fourth update message carrying the first capability set; A fifth update message is sent to the first terminal, the fifth update message being used to request the first terminal to establish multiple media paths, the multiple media paths including media paths based on the first terminal's front-facing camera correlation capabilities and rear-facing camera correlation capabilities; When the first terminal completes the establishment of the multi-channel media path, it receives the fourth response message sent by the first terminal and establishes a second session channel with the first terminal based on the multi-channel media path.

17. The method according to claim 16, characterized in that, Sending the second instruction information to the second terminal includes: Send an eighth update message to the second terminal, wherein the second indication information is carried in the eighth update message, and the eighth update message is used to instruct the second terminal to respond to the second session initiated by the first terminal when the multi-channel media path is established; Panoramic composite information is sent to the first terminal based on the second session channel.

18. A media negotiation method, characterized in that, Applied to a second terminal, the method includes: Receive a first session initiated by a first terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability; Send a second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal; If the first expected media capability and the second expected media capability do not match, the first expected media capability shall be reported to the server. Receive a second indication information sent by the server, the second indication information being used to indicate a second media negotiation method, the second media negotiation method being determined based on the first expected media capability; The second session initiated by the first terminal is responded to using the second media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

19. The method according to claim 18, characterized in that, The second expected media capability includes at least one of the second terminal's panoramic encoding capability and panoramic decoding capability.

20. The method according to claim 19, characterized in that, The first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capabilities and the rear-facing camera-related capabilities of the first terminal, the first expected media capability is the panoramic decoding capability of the first terminal. The second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal. The receipt of the second indication information sent by the server includes: Receive the sixth update message sent by the server, wherein the second indication information is carried in the sixth update message; The sixth update message is used to respond to the second session initiated by the first terminal based on the first session channel when a first session channel is established between the first terminal and the server.

21. The method according to claim 19, characterized in that, The first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability and the rear-facing camera-related capability of the first terminal, the first expected media capability in the first session is the panoramic decoding capability of the first terminal, and the second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal. The receipt of the second indication information sent by the server includes: Send a second update message to the server, the second update message being used to request the first terminal to update the first expected media capability to the first terminal's front-facing camera-related capability and / or rear-facing camera-related capability; When the server completes the dual-channel resource release, a seventh update message is received from the server, and the second indication information is carried in the seventh update message; The seventh update message is used to respond to the second session initiated by the first terminal based on the updated first expected media capability, wherein the updated first expected media capability is the front camera related capability and the rear camera related capability of the first terminal.

22. The method according to claim 19, characterized in that, The first media capability information includes a first capability set. When the first capability set includes the front-facing camera-related capability, the rear-facing camera-related capability, and the panoramic encoding capability of the first terminal, the first expected media capability in the first session is the front-facing camera-related capability and / or the rear-facing camera-related capability of the first terminal. The second expected media capability includes the panoramic encoding capability and the panoramic decoding capability of the second terminal. The receipt of the second indication information sent by the server includes: Send a third update message to the server, the third update message being used to request the first terminal to update the first expected media capability to the first terminal's panoramic decoding capability; The first terminal receives a third response message sent by the server. The third response message is transmitted from the first terminal to the server and carries the updated first expected media capability, which is the panoramic decoding capability of the first terminal. When the first terminal and the server complete the establishment of the second session channel, the server sends an eighth update message, in which the second indication information is carried; wherein the eighth update message is used to respond to the second session initiated by the first terminal.

23. The method according to claim 19, characterized in that, Before receiving the first session initiated by the first terminal, the method further includes: Send a second capability set to the server; the second capability set includes at least one of the front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic camera-related capabilities of the second terminal.

24. A media negotiation device, characterized in that, Applied to a first terminal, the device includes: The first session module is used to initiate a first session to the second terminal. The first session carries first media capability information, which includes a first expected media capability. The first reporting module is used to report the second expected media capability to the server when the first expected media capability does not match the second expected media capability sent by the second terminal. A first receiving module is configured to receive first indication information sent by the server, the first indication information being used to indicate a first media negotiation method, the first media negotiation method being determined based on the second expected media capability; The second session module is used to initiate a second session with the second terminal using the first media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

25. A media negotiation device, characterized in that, Applied to a server, the device includes: The second receiving module is used to receive first media capability information and second media capability information sent by the first terminal and the second terminal respectively. The first media capability information includes a first expected media capability, and the second media capability information includes a second expected media capability. The negotiation determination module is used to determine a first media negotiation method based on the second expected media capability, and to determine a second media negotiation method based on the first expected media capability information; The first sending module is configured to send first indication information to the first terminal, the first indication information being used to indicate the first media negotiation method, and to send second indication information to the second terminal, the second indication information being used to indicate the second media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

26. A media negotiation device, characterized in that, Applied to a second terminal, the device includes: The third receiving module is used to receive a first session initiated by the first terminal, the first session carrying first media capability information, the first media capability information including a first expected media capability; The second sending module is used to send a second expected media capability to the first terminal, wherein the second expected media capability is the expected media capability of the second terminal; The second reporting module is used to report the first expected media capability to the server when the first expected media capability and the second expected media capability do not match. The fourth receiving module is used to receive second indication information sent by the server, the second indication information being used to indicate a second media negotiation method, the second media negotiation method being determined based on the first expected media capability; The session response module is used to respond to the second session initiated by the first terminal using the second media negotiation method; The first media capability information includes a first capability set, which includes at least one of the first terminal's front-facing camera-related capabilities, rear-facing camera-related capabilities, and panoramic decoding capabilities. The first expected media capability includes at least one of the following: front-facing camera-related capability, rear-facing camera-related capability, and panoramic decoding capability of the first terminal.

27. A communication device, comprising: A transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; characterized in that, The transceiver is configured to implement the steps of the method as described in any one of claims 1 to 8; or to implement the steps of the method as described in any one of claims 9 to 17; or to implement the steps of the method as described in any one of claims 18 to 23.

28. A computer-readable storage medium for storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8; or implements the steps of the method as described in any one of claims 9 to 17; or implements the steps of the method as described in any one of claims 18 to 23.

29. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 8; or implement the steps of the method as claimed in any one of claims 9 to 17; or implement the steps of the method as claimed in any one of claims 18 to 23.

Images