5G new call screen sharing method, device and system based on ICE protocol

Abstract

The invention discloses a 5G new call screen sharing method, device and system based on an ICE protocol. The method comprises the following steps: establishing an audio call based on an AS, an audio data interaction channel between an MS and calling and called terminals, and a first video data interaction channel from the MS to the called terminal; establishing a second video data interaction channel from the calling terminal to the MS based on the ICE protocol; the calling terminal obtains a to-be-shared data packet according to the local screen information and sends the to-be-shared data packet to the MS through a second video data interaction channel; the MS performs data compatibility processing according to the video format standard of the IMS communication network, generates a to-be-displayed data packet and sends the to-be-displayed data packet to the called terminal through the first video data interaction channel; and the called terminal displays the screen information of the calling terminal in real time according to the data packet to be displayed. According to the invention, information interaction and real-time screen sharing between the terminal internet side and the IMS communication network can be realized without upgrading media network elements and terminal side hardware and downloading applets.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a new 5G call screen sharing method, apparatus, computer-readable storage medium, computer program product, media server and system based on the ICE protocol. Background Technology

[0002] Currently, basic calls provided by operators rely solely on voice for communication, which has limitations such as limited functionality and content. 5G (5th Generation Mobile Communication Technology) represents a technological and business upgrade to basic calls, overcoming these limitations to provide users with a new multimedia, visual, and fully interactive experience. Real-time screen sharing during a call is a typical application scenario, allowing for direct display of content without the need for detailed explanations of complex information via voice or text. This simplifies communication, facilitates remote guidance, and enables rapid problem-solving.

[0003] The fundamental problem to be solved in the real-time screen sharing scenario of 5G new calls is how to transmit information from the terminal's intranet to the IMS (IP Multimedia Subsystem) communication network. The existing solution is to establish a data channel (DC) and upgrade the basic media network element to a media network element with the VoNR+ (Voice over NewRadio Plus) technical architecture. The upgraded media network element adds capabilities such as DC signaling negotiation, DC channel creation, and DC applet download. When establishing a call, the terminal performs signaling negotiation, creates a DC channel, and downloads the DC applet. During the call, the user operates the applet to achieve screen sharing.

[0004] Although the existing solution uses the DC channel as a bridge to connect the terminal's intranet side and the IMS communication network, it has the following disadvantages: (1) Upgrading media network elements will affect existing services and involve high costs, including equipment costs, implementation costs, training costs and maintenance costs; (2) Terminal devices need to be upgraded in hardware before leaving the factory to support the new DC capabilities, which not only increases production costs, but also requires solving compatibility issues between various mobile phone manufacturers. At the same time, existing terminal devices also need to be gradually replaced to support new functions, which will result in the coverage rate not reaching the expected level in the short term; (3) Each call requires downloading the DC applet, which will consume additional bandwidth and affect the overall performance of the network. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, computer-readable storage medium, computer program product, media server, and system for 5G new call screen sharing based on the ICE protocol. This enables information interaction between the terminal's Internet side and the IMS communication network, as well as real-time screen sharing during 5G new calls, without requiring upgrades to media network elements, terminal hardware, or the download of a mini-program during the call.

[0006] To achieve the above objectives, embodiments of the present invention provide a 5G new call screen sharing method based on the ICE protocol, applied to a media server, comprising: The application server receives a video session creation request and establishes a first video data interaction channel between the media server and the called terminal. The video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal, as well as the audio call between the calling terminal and the called terminal, are established. Based on the ICE protocol, a second video data interaction channel is established between the calling terminal and the media server; The second video data interaction channel receives a data packet to be shared sent by the calling terminal; wherein the data packet to be shared is obtained by the calling terminal after processing its local screen information and then sent to the media server. The received data packets to be shared are processed for data recovery, and the recovered data is processed for compatibility according to the video format standard of the IMS communication network to generate data packets to be displayed. The generated data packet to be displayed is sent to the called terminal through the first video data interaction channel, so that the called terminal can display the screen information of the calling terminal in real time according to the received data packet to be displayed.

[0007] To achieve the above objectives, embodiments of the present invention also provide a 5G new call screen sharing device based on the ICE protocol, which is installed in a media server and includes: The first video channel establishment module is used to receive a video session creation request sent by the application server and establish a first video data interaction channel from the media server to the called terminal. The video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal, as well as the audio call between the calling terminal and the called terminal, are established. The second video channel establishment module is used to establish a second video data interaction channel between the calling terminal and the media server based on the ICE protocol. The data packet receiving module is used to receive the data packet to be shared sent by the calling terminal through the second video data interaction channel; wherein the data packet to be shared is obtained by the calling terminal after processing the local screen information and then sent to the media server; The data packet generation module is used to perform data recovery processing on the received data packets to be shared, and to perform compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate the data packets to be displayed. The data packet to be displayed module is used to send the generated data packet to be displayed to the called terminal through the first video data interaction channel, so that the called terminal can display the screen information of the calling terminal in real time according to the received data packet to be displayed.

[0008] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the 5G new call screen sharing method based on the ICE protocol described above.

[0009] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the 5G new call screen sharing method based on the ICE protocol described above.

[0010] This invention also provides a media server, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the 5G new call screen sharing method based on the ICE protocol described above.

[0011] This invention also provides a 5G new call screen sharing system based on the ICE protocol, including a calling terminal, a called terminal, an application server, and a media server. The media server is used to cooperate with the calling terminal, the called terminal, and the application server to implement the 5G new call screen sharing method based on the ICE protocol described above.

[0012] Compared with existing technologies, the embodiments of the present invention provide a method, apparatus, computer-readable storage medium, computer program product, media server, and system for 5G new call screen sharing based on the ICE protocol. By establishing a first video data interaction channel from the mobile terminal (MS) to the called terminal, and establishing a second video data interaction channel from the calling terminal to the MS based on the ICE protocol, the data packets to be shared from the calling terminal are received through the second video data interaction channel. After data recovery and IMS format compatibility processing, the data is then sent to the called terminal for real-time display through the first video data interaction channel. This enables information interaction between the terminal's internet side and the IMS communication network, as well as real-time screen sharing during 5G new calls. Furthermore, it eliminates the need to upgrade media network elements, upgrade terminal hardware, or download mini-programs during calls, thereby reducing various upgrade and modification costs, improving terminal compatibility and coverage, and simultaneously reducing network bandwidth consumption and improving overall network performance. Attached Figure Description

[0013] Figure 1 This is a flowchart of a new 5G call screen sharing method based on the ICE protocol provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of a 5G new call screen sharing system based on the ICE protocol provided in an embodiment of the present invention; Figure 3 This is a flowchart of a call signaling interaction provided in an embodiment of the present invention; Figure 4 This is a data interaction flowchart based on the ICE protocol provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a 5G new call screen sharing device based on the ICE protocol provided in an embodiment of the present invention; Figure 6 This is a structural block diagram of a media server provided in an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] This invention provides a new 5G call screen sharing method based on the ICE protocol. See [link to relevant documentation]. Figure 1The diagram shows a flowchart of a 5G new call screen sharing method based on the ICE protocol according to an embodiment of the present invention. The method is applied to a media server (MS) and executed by the MS. The method includes steps S11 to S15: Step S11: Receive a video session creation request sent by the application server and establish a first video data interaction channel from the media server to the called terminal; wherein, the video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal, as well as the audio call between the calling terminal and the called terminal, are established.

[0016] Specifically, after establishing audio data interaction channels between the MS and the calling terminal, between the MS and the called terminal, and between the calling and called terminals based on the application server (AS), the AS will initiate a video session creation request to the MS based on the audio call. The MS is used to receive the video session creation request sent by the AS, and after receiving the video session creation request sent by the AS, it establishes the first video data interaction channel from the MS to the called terminal.

[0017] It should be noted that, since a video data interaction channel (i.e., the second video data interaction channel hereinafter referred to as the second video data interaction channel) between the calling terminal and the MS will be established based on the ICE (Interactive Connectivity Establishment) protocol, this embodiment of the invention modifies the existing bilateral video call into a one-sided video call. That is, only a video data interaction channel (i.e., the first video data interaction channel) between the MS and the called terminal is established, and no video data interaction channel between the MS and the calling terminal is established.

[0018] Step S12: Based on the ICE protocol, establish a second video data interaction channel between the calling terminal and the media server.

[0019] It should be noted that the ICE protocol is an end-to-end standard communication protocol, mainly used in Web Real-Time Communications (WebRTC). However, 5G calls are traditional operator calls, with data interaction conducted in a proprietary IMS communication network. It is not suitable to directly introduce WebRTC. Therefore, considering the real-time screen sharing feature of 5G calls, this embodiment of the invention not only upgrades the voice call to a one-sided video call between the MS and the called terminal (step S11), but also adds a process for establishing a video data interaction channel (i.e., a second video data interaction channel) between the calling terminal and the MS through the ICE protocol (step S12). This design can guarantee the basic voice call function and complete the system upgrade at the software level without affecting the original business and media network element architecture.

[0020] Step S13: Receive the data packet to be shared sent by the calling terminal through the second video data interaction channel; wherein the data packet to be shared is obtained by the calling terminal after processing the local screen information and sent to the media server.

[0021] Specifically, after the first video data interaction channel from the MS to the called terminal and the second video data interaction channel from the calling terminal to the MS are established, the calling terminal can capture local screen information, and after processing the captured screen information, send the obtained data packet to be shared to the MS through the second video data interaction channel. Correspondingly, the MS can receive the data packet to be shared sent by the calling terminal through the second video data interaction channel.

[0022] Step S14: Perform data recovery processing on the received data packets to be shared, and perform compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate data packets to be displayed.

[0023] Specifically, after receiving the data packet to be shared sent by the calling terminal through the second video data interaction channel, the MS can perform data recovery processing on the received data packet to obtain complete video frame data, and perform compatibility processing on the recovered complete video frame data according to the video format standard of the IMS communication network to generate the corresponding data packet to be displayed.

[0024] Understandably, since the data packet to be shared is obtained by the calling terminal after processing the local screen information, the MS can parse the data packet to be shared according to the opposite rules of the calling terminal's processing flow when performing data recovery processing, thereby obtaining complete video frame data.

[0025] Step S15: The generated data packet to be displayed is sent to the called terminal through the first video data interaction channel, so that the called terminal displays the screen information of the calling terminal in real time according to the received data packet to be displayed.

[0026] Specifically, after generating the data packet to be displayed, the MS can send the generated data packet to the called terminal through the first video data interaction channel via the IMS communication network. The called terminal is used to receive the data packet to be displayed sent by the MS, and after receiving the data packet to be displayed sent by the MS, it restores the received data packet to the screen information shared by the calling terminal and displays it in real time, thereby completing real-time screen sharing.

[0027] See Figure 2 The diagram shown is a structural block diagram of a 5G new call screen sharing system based on the ICE protocol according to an embodiment of the present invention. The system includes a calling terminal, a called terminal, an AS (Autopilot), and an MS (Mobile Switch). The calling terminal has a software development kit (SDK) pre-embedded in it, and the SDK integrates functions such as the ICE protocol and encoding. Furthermore... Figure 2 The system shown can also serve as a typical application scenario example for real-time screen sharing in 5G new calls (the calling terminal initiates screen sharing, and the called terminal receives and displays the shared screen information). In the real-time screen sharing scenario of 5G new calls, the AS interacts with the calling terminal, the called terminal, and the MS to establish audio data interaction channels between the MS and the calling terminal, and between the MS and the called terminal; the AS interacts with the calling terminal and the called terminal using SDP (Session Description Protocol) signaling. The calling terminal establishes an audio call with the called terminal using a session description protocol (SMR) to ensure basic call functionality. The AS initiates a video session creation request to the MS to establish the first video data interaction channel between the MS and the called terminal, thus realizing information interaction between the called terminal's internet side and the IMS communication network. The calling terminal activates the SDK and negotiates with the MS via ICE to establish a second video data interaction channel between the calling terminal and the MS, thus realizing information interaction between the calling terminal's internet side and the IMS communication network. At this point, the calling terminal can capture local screen information, process it, and send it to the MS via the IMS communication network. The MS processes the data received from the calling terminal and sends it to the called terminal via the IMS communication network. The called terminal can then display the screen information shared by the calling terminal in real time, thus achieving real-time screen sharing during the call.

[0028] It should be noted that the ICE protocol is widely used, especially in the field of audio and video transmission. In existing applications, end-to-end communication is established through the ICE protocol to establish a direct data exchange channel for real-time communication, such as WebRTC. Media stream exchange channels are established by negotiating SDP signaling (candidate information, media format, control information, etc.). However, this negotiation information is very cumbersome and the negotiation process is very complex. For example, incompatibility between audio and video formats can trigger multiple negotiations. Furthermore, the management of the communication process is also very complicated, such as the joining and leaving of members in a multi-person communication process. At the same time, the communication network is required to be open, such as a general Internet channel.

[0029] The 5G new call real-time screen sharing scenario has its own unique features: "New call" is a traditional operator call, with data interaction conducted in a proprietary IMS communication network, which is not suitable for directly introducing WebRTC; the call only involves the calling and called ends, so there is no need for such complicated communication process management; in this scenario, the calling terminal transmits screen information and the called terminal displays screen information, so there is no need to establish a two-way ICE channel; this scenario relies on the IMS communication network, and the video format standard is relatively fixed, so there is no need to consider the situation where multiple negotiations are required due to media format incompatibility.

[0030] This invention provides a 5G new call screen sharing method based on the ICE protocol. It establishes a first video data interaction channel from the mobile terminal (MS) to the called terminal, and a second video data interaction channel from the calling terminal to the MS based on the ICE protocol. The method receives the data packet to be shared from the calling terminal through the second video data interaction channel. After data recovery and IMS format compatibility processing, the data is then sent to the called terminal through the first video data interaction channel for real-time display. This enables information interaction between the terminal's internet side and the IMS communication network, as well as real-time screen sharing during 5G new calls. Furthermore, it eliminates the need to upgrade media network elements, terminal hardware, or download mini-programs during calls, thereby reducing upgrade and modification costs, improving terminal compatibility and coverage, and simultaneously reducing network bandwidth consumption and improving overall network performance.

[0031] In one optional embodiment, the application server is used to interact with the calling terminal, the called terminal, and the media server to establish audio data interaction channels between the media server and the calling terminal, and between the media server and the called terminal; it is also used to receive and transmit signaling between the calling terminal and the called terminal through a session description protocol to establish an audio call between the calling terminal and the called terminal.

[0032] Specifically, in combination Figure 2As in the above embodiments, the AS interacts with the calling terminal, the called terminal, and the MS respectively to establish an audio data interaction channel between the MS and the calling terminal, and between the MS and the called terminal. Furthermore, the AS receives and transmits signaling between the calling terminal and the called terminal through the Session Description Protocol (SDP) to establish an audio call between the calling terminal and the called terminal.

[0033] See Figure 3 The diagram shown is a call signaling interaction flowchart provided in an embodiment of the present invention. Figure 3 This mainly demonstrates the interaction process of establishing audio call and audio / video data exchange channels, the interaction process of screen sharing, and the interaction process of session termination. Signaling from both the calling and called terminals is routed to the call service AS. The interaction process includes: The first phase involves establishing an audio call + audio data interaction channel between the MS and the calling and called parties. This mainly involves completing the audio signaling negotiation between the calling terminal, AS, and called terminal, and simultaneously establishing audio data interaction channels between the MS and the calling terminal and between the MS and the called terminal. Specifically, the calling terminal initiates an audio session request carrying SDP to the AS, which is then transparently transmitted to the called terminal, enabling the called terminal to respond to the audio session request. At the same time, the AS triggers the establishment of the audio data interaction channel between the MS and the calling and called terminals. Furthermore, the AS receives and transparently transmits signaling between the calling and called terminals, and the called terminal triggers the 180 Ringing→PRACK→200 OK→ACK signaling process to complete the audio call establishment. The second stage involves establishing a one-way video channel from the MS to the called terminal. This mainly involves the AS initiating a video session creation request to the MS based on the audio call, and establishing the first video data interaction channel between the MS and the called terminal through session upgrade. The third stage, screen sharing data transmission, mainly involves the calling terminal capturing local screen information, processing it, and then sending it to the MS via the second video data interaction channel from the calling terminal to the MS (the establishment process of the second video data interaction channel will be described in detail in subsequent embodiments). The MS then processes the received data and sends it to the called terminal via the first video data interaction channel from the MS to the called terminal, enabling the called terminal to display the screen information shared by the calling terminal in real time. Figure 3 The "Data" interaction in the text corresponds to this process. The fourth stage, session termination, takes the called terminal requesting to end the session as an example, as follows: The called terminal sends a BYE signaling message (requesting to end the session) to the AS, which is then transparently transmitted to the calling terminal by the AS; after receiving the BYE signaling message, the calling terminal returns a 200 OK signaling message to the AS to acknowledge the BYE signaling message, which is then transparently transmitted to the called terminal by the AS; the AS also initiates a Stop Session(all) request to the MS and carries the session status Status(all); after receiving Stop Session(all), the MS closes the audio and video data interaction channels with the calling terminal and the called terminal, and the entire session is terminated.

[0034] The present invention provides a 5G new call screen sharing method based on the ICE protocol. By first establishing the audio call and then starting the video data interaction channel required for screen sharing, the method can uniformly manage the establishment sequence of the audio and video data interaction channels, avoid audio and video data interaction channel conflicts, simplify the signaling interaction link, reduce the signaling processing load of the MS, and improve the stability of audio and video collaboration.

[0035] In one optional embodiment, establishing a second video data interaction channel from the calling terminal to the media server based on the ICE protocol specifically includes: Based on the collected local addresses and the server reflection addresses and relay addresses obtained from the STUN / TURN server, a first candidate list is established; wherein, the first candidate list includes host candidates, server reflection candidates and relay candidates sorted by priority from high to low, as well as the priority index of each candidate. Obtain a second candidate list established by the calling terminal; wherein the second candidate list is established by the calling terminal based on the collected local address and the server reflection address and trunk address obtained from the STUN / TURN server, using the same sorting rules as the first candidate list; Multiple candidate pairs are formed based on the first candidate list and the second candidate list, and a priority index is calculated for each candidate pair; wherein each candidate pair includes one candidate from the first candidate list and one candidate from the second candidate list; ICE connectivity checks are performed on each candidate pair in descending order of priority index until a candidate pair with connectivity is obtained, and the obtained candidate pair with connectivity is taken as a usable candidate pair; wherein, the usable candidate pair is the same as the candidate pair with connectivity obtained by the calling terminal based on the first candidate list and the second candidate list; A second video data interaction channel is established between the calling terminal and the media server based on the available candidate pairs.

[0036] It should be noted that, in order to facilitate understanding of the specific content of the embodiments of the present invention, the meanings of several technical terms will be explained below: Agent: According to the definition in RFC3264, it refers to an implementation of an offer and answer interaction protocol, which in this embodiment of the invention can be equivalent to the calling terminal and MS.

[0037] Peer: From the perspective of one agent in a session, its peer agent is another agent. In the embodiments of this invention, it can be equivalent to the calling terminal and the MS being each other's peer agents.

[0038] Transport Address: A combination of IP address (Internet Protocol Address), transport protocol (such as UDP (User Datagram Protocol) or TCP (Transmission Control Protocol)) and port.

[0039] Candidate: A transport address, which is a potential contact point for receiving media, and also includes attributes such as type, priority, base, and base address.

[0040] Host Candidate: A candidate obtained by binding to a specific port on an IP address on a host, including IP addresses on physical interfaces and IP addresses on logical interfaces, such as IP addresses obtained through VPN (Virtual Private Network) and RSIP (Domain Specific IP) [RFC3102] (which exist at the operating system level).

[0041] Server Reflexive Candidate: When a proxy sends packets to a server via NAT (Network Address Translation), the NAT assigns a bound IP address and port to the proxy. Server reflexive candidates can be obtained by using a STUN (Session Traversal Utilities for NAT) server or a TURN (Traversal Using Relays around NAT) server (which provides relay and server reflexive candidates) that makes the binding request.

[0042] Peer Reflexive Candidate: The binding IP address and port assigned to the proxy by the NAT when the proxy sends a STUN binding request to its peer through the NAT.

[0043] Relayed Candidate: A candidate obtained by sending a TURN allocation request from the host candidate to the TURN server. The relayed candidate resides on the TURN server, and the TURN server relays the data packets back to the broker.

[0044] Local Candidate: A candidate that the agent acquires and includes in the SDP Offer or SDP Offer it sends.

[0045] Remote Candidate: A candidate received by the agent from the peer's SDP Offer or SDP Answer.

[0046] Candidate Pair: A pair consisting of a local candidate and a remote candidate.

[0047] Specifically, in conjunction with the above embodiments, when the MS establishes a second video data interaction channel from the calling terminal to the media server based on the ICE protocol, assuming that a STUN / TURN server (STS) and a SIP server (Session Initiation Protocol Server) have been deployed in the public network environment, and the calling terminal has embedded the SDK, then the MS can first use the collected local address of the MS and the server reflection address and relay address obtained from the STS. Address), and establish a first candidate list, which includes host candidates, server reflection candidates, and relay candidates sorted by priority from high to low (i.e., host candidates have higher priority than server reflection candidates, which in turn have higher priority than relay candidates), and also includes the priority index for each candidate. Then, obtain a second candidate list established by the calling terminal from the SIPS. It should be noted that the second candidate list is established by the calling terminal based on its local address, the server reflection address, and the relay address obtained from the STS, using the same sorting rules as the first candidate list. Afterwards, the MS can pair candidates based on the first candidate list (equivalent to the MS's local candidate list) and the second candidate list (equivalent to the MS's remote candidate list), forming multiple candidate pairs, such that each candidate pair includes one candidate from the first candidate list. The calling terminal selects one candidate from the first candidate list (equivalent to the local candidate list of the MS) and one candidate from the second candidate list (equivalent to the remote candidate list of the MS), and calculates the priority index for each candidate pair. Then, the MS can perform ICE connectivity checks on each candidate pair in descending order of priority index until a candidate pair with connectivity is found, and the found candidate pair with connectivity is used as a usable candidate pair. At the same time, the calling terminal will also adopt a similar processing flow as the MS, find a candidate pair with connectivity based on the first candidate list (equivalent to the remote candidate list of the calling terminal) and the second candidate list (equivalent to the local candidate list of the calling terminal), and the candidate pair with connectivity found by the calling terminal is consistent with the candidate pair with connectivity found by the MS, that is, it is the same usable candidate pair. Therefore, based on the usable candidate pair, the second video data interaction channel from the calling terminal to the MS can be established, thereby ensuring the connectivity between the calling terminal and the MS.

[0048] Understandably, the worst-case scenario of the ICE connectivity check is that the available candidate pairs found are candidates where both the local and remote ends are relay candidates (lowest priority), but connectivity from the calling terminal to the MS can still be guaranteed.

[0049] The present invention provides a 5G new call screen sharing method based on the ICE protocol. This method involves constructing a first candidate list containing candidates with different priorities using a mobile terminal (MS), obtaining a second candidate list established by the calling terminal, forming corresponding candidate pairs, calculating the priority of each pair, and then performing ICE connectivity checks according to priority from high to low to determine available candidate pairs. This establishes a second video data interaction channel, covering network scenarios involving three types of candidates: hosts, server reflections, and relays, thereby improving the NAT traversal success rate. Simultaneously, by prioritizing and selecting high-quality paths, invalid connection attempts are reduced, thus lowering the establishment latency of the second video data interaction channel.

[0050] In one optional embodiment, the step of establishing a first candidate list based on the collected local addresses and the server reflection addresses and relay addresses obtained from the STUN / TURN server specifically includes: Collect local addresses and send requests to the STUN / TURN server to obtain the server reflection address and relay address; The host candidates corresponding to the local address, the server reflection candidates corresponding to the server reflection address, and the relay candidates corresponding to the relay address are sorted in descending order of priority index to form the first candidate list.

[0051] Specifically, in conjunction with the above embodiments, when the MS establishes a first candidate list based on the collected local addresses and the server reflection addresses and relay addresses obtained from the STS, it can first collect the MS's local addresses according to the acquisition order of each candidate (the acquisition order can be determined based on the address acquisition logic defined by the definition of each candidate), send requests to the STS to obtain the server reflection addresses and relay addresses, and form corresponding host candidates (corresponding to local addresses), server reflection candidates (corresponding to server reflection addresses), and relay candidates (corresponding to relay addresses), respectively. Then, according to the priority indicators of the host candidates, the server reflection candidates, and the relay candidates, the candidates are sorted in descending order of their priority indicators to form the first candidate list. It can be understood that the earlier the candidate is sorted, the higher its priority, which is more conducive to data transmission.

[0052] The present invention provides a 5G new call screen sharing method based on the ICE protocol. This method involves the MS collecting local addresses, obtaining server reflection addresses and relay addresses from the STS, and sorting the corresponding candidates from high to low priority to construct a first candidate list. This standardizes the construction rules of the candidate list, ensuring consistency in the sorting logic between the MS and the calling terminal's second candidate list. Simultaneously, it ensures that during subsequent candidate pairing and ICE connectivity checks, the host candidate corresponding to the local address is prioritized, balancing connection establishment speed and network adaptability, thereby improving channel establishment efficiency.

[0053] In one optional embodiment, obtaining the second candidate list established by the calling terminal specifically includes: Construct the first SDP Offer information based on the first candidate list, username, password, and room number; The first SDP Offer information is sent to the SIP server, which parses the received SDP Offer information to obtain the room number and determines whether the second SDP Offer information sent by the calling terminal with the same room number already exists in the cache. If so, the first SDP Offer information is constructed based on the second SDP Offer information and returned to the media server. The second SDP Offer information is constructed by the calling terminal based on the established second candidate list, username, password and room number and then sent to the SIP server. Receive and parse the first SDP Answer information to obtain the second candidate list carried in the first SDP Answer information.

[0054] Specifically, in conjunction with the above embodiments, when the MS obtains the second candidate list established by the calling terminal, it can, after establishing the first candidate list, construct the first SDP Offer information based on the information such as each candidate, username, password, and room number contained in the first candidate list, and initiate a request to the SIPS based on the first SDP Offer information. Simultaneously, the calling terminal will also adopt a similar processing flow as the MS, constructing the second SDP Offer information based on the information such as each candidate, username, password, and room number contained in the second candidate list, and initiating a request to the SIPS based on the second SDP Offer information. After receiving the request from the MS, the SIPS parses the received first SDP Offer information to obtain the information such as each candidate, username, password, and room number carried therein, and checks whether the second SDP Offer information sent by the calling terminal already exists in the SIPS cache and whether the second SDP Offer information carries the same room number (i.e., whether the room number sent by the MS is the same as the room number sent by the calling terminal). If so, it uses the SDP Offer information sent by the peer (i.e., the calling terminal) in the cache (i.e., the second SDP Offer). Based on the offer information, the first SDP answer information is constructed and returned to the MS. Similarly, after receiving the request initiated by the calling terminal, the SIPS obtains the information such as candidates, usernames, passwords, and room numbers carried in the received second SDP offer information by parsing it. It also checks whether the first SDP offer information sent by the MS already exists in the SIPS cache and whether the first SDP offer information carries the same room number (i.e., whether the room number sent by the calling terminal is the same as the room number sent by the MS). If so, the second SDP answer information is constructed based on the SDP offer information sent by the peer (i.e., the MS) in the cache (i.e., the first SDP offer information) and returned to the calling terminal. The MS receives the first SDP answer information returned by the SIPS and parses it to obtain the second candidate list carried in the second SDP offer information corresponding to the first SDP answer information.

[0055] It should be noted that for SIPS, regardless of whether it receives a request from the MS or a request from the calling terminal, it will parse the SDP Offer information carried in the received request and check whether the same room number sent by the peer exists. If it exists, it will construct the SDP Answer information based on the SDP Offer information sent by the peer. If it does not exist, it means that SIPS has not yet received the SDP Offer information sent by the peer. Therefore, it can first create a new room with the currently received room number as the key and cache it (i.e., cache the currently received SDP Offer information) and wait for the SDP Offer information sent by the peer before proceeding with subsequent processing.

[0056] It should be noted that the above processing flow illustrates how to establish a second video data interaction channel between the calling terminal and the MS through a single SDP negotiation. Based on the special characteristics of the real-time screen sharing scenario in 5G new calls, this embodiment of the invention has made targeted designs for the ICE negotiation process, removing complex media information and only using a simple format to load necessary information (such as room number, role (i.e., candidate), username, etc.). The complex media negotiation process has been eliminated, so the exchange of SDP information can be completed in one negotiation, thereby improving the stability, scalability and flexibility of the system.

[0057] In one optional embodiment, forming multiple candidate pairs based on the first candidate list and the second candidate list specifically includes: According to the order of each candidate in the first candidate list and the order of each candidate in the second candidate list, two candidates in the first candidate list and the second candidate list that are in the same order position are paired up to form multiple candidate pairs.

[0058] Specifically, in conjunction with the above embodiments, when the MS forms multiple candidate pairs based on the first candidate list and the second candidate list, it can sort the candidates in the first candidate list by priority and the candidates in the second candidate list by priority, and pair up two candidates in the first candidate list and the second candidate list that are in the same sorting position respectively. That is, pair the candidate in the first candidate list that is in the first priority with the candidate in the second candidate list that is in the first priority, pair the candidate in the first candidate list that is in the second priority with the candidate in the second priority, pair the candidate in the first candidate list that is in the third priority with the candidate in the second candidate list that is in the third priority, and so on, until all pairings are completed between the first candidate list and the second candidate list, thus forming multiple candidate pairs.

[0059] The present invention provides a 5G new call screen sharing method based on the ICE protocol. According to the priority of each candidate in the first candidate list and the second candidate list, the candidates with the same ranking position are semantically paired to form candidate pairs. This can reduce the number of candidate pair pairings, avoid the redundant calculation caused by full pairing, thereby reducing the processing overhead of the MS. It can also ensure that high-priority candidates are paired first, thereby increasing the probability of selecting high-quality network paths.

[0060] In one optional embodiment, the priority index of each candidate is a preset quantification value, and the larger the preset quantification value, the higher the priority of the candidate.

[0061] Specifically, in conjunction with the above embodiments, the priority index of each candidate in the first candidate list and the second candidate list can be a preset quantification value. That is, the priority of the candidate is represented by the preset quantification value (pre-set preference value), and the larger the preset quantification value of the candidate, the higher its corresponding priority.

[0062] For example, the preset quantization value for host candidates is 126, the preset quantization value for server reflection candidates is 100, and the preset quantization value for relay candidates is 0. It can be understood that the larger the value, the higher the priority, the earlier the ranking in the list, and the more beneficial it is for data transmission.

[0063] The present invention provides a 5G new call screen sharing method based on the ICE protocol. By setting a preset quantization value as a priority indicator for each candidate, the higher the quantization value, the higher the priority. This method can quantify and standardize the priority of candidates, avoiding confusion caused by subjective ranking. At the same time, the quantization value can be adjusted to adapt to different network scenarios (such as increasing the quantization value of relay candidates in weak network conditions), thereby enhancing the flexibility of channel establishment.

[0064] In one alternative embodiment, the priority index of each candidate pair is the product of the preset quantization values ​​of the two candidates contained in the candidate pair.

[0065] Specifically, in conjunction with the above embodiments, under the premise that the priority index of the candidates is a preset quantization value, when the MS calculates the priority index of each candidate pair, it can directly calculate the product of the preset quantization values ​​of the two candidates contained in each candidate pair, and use the calculated product as the priority index of each candidate pair.

[0066] The present invention provides a 5G new call screen sharing method based on the ICE protocol. By multiplying the preset quantization values ​​of the two candidates in a candidate pair and using the product as a priority index for the candidate pair, the quality of local and remote candidates can be comprehensively evaluated. Candidate pairs with high priority at both ends are selected first, thereby improving the transmission stability of the channel. Furthermore, by quantifying the overall quality of the candidate pair through numerical product, the order of subsequent connectivity checks can be made more reasonable.

[0067] In one optional embodiment, the candidates in the first candidate list included in each candidate pair are designated as local candidates, and the candidates in the second candidate list are designated as remote candidates. The step of performing ICE connectivity checks on each candidate pair in descending order of priority index specifically includes: Following the traversal order of priority indicators from largest to smallest, for each candidate pair, a STUN binding request is sent from the local candidate to the remote candidate. If no STUN binding response is received from the remote candidate after multiple requests within a preset time period, the corresponding candidate pair is determined to be unconnected; otherwise, the corresponding candidate pair is determined to be connected.

[0068] Specifically, in conjunction with the above embodiments, when the MS performs ICE connectivity checks on each candidate pair in descending order of priority, it can use its own first candidate list as the local candidate list and the calling terminal's second candidate list as the remote candidate list. The candidates in the first candidate list included in each candidate pair are local candidates, and the candidates in the second candidate list included in each candidate pair are remote candidates. Based on this, the candidate pairs are traversed sequentially in descending order of priority. For each candidate pair, during ICE connectivity checks, a STUN binding request can be sent from the local candidate to the remote candidate. If a STUN binding response is not received from the remote candidate after reconnection within a preset time period, the corresponding candidate pair is determined to lack connectivity, and the ICE connectivity check continues for the next candidate pair until a connected candidate pair is found, at which point the check stops. If a STUN binding response is received from the remote candidate within the preset time period, the corresponding candidate pair is determined to have connectivity, and the check stops.

[0069] The present invention provides a 5G new call screen sharing method based on the ICE protocol, which traverses candidate pairs in descending order of priority, and sends a STUN binding request from the local candidate to the remote candidate in the candidate pair. The connectivity of the candidate pair is determined based on whether a response is received within a specified time. Based on the standardized request-response mechanism of the STUN protocol, the connectivity of the candidate pair can be accurately determined, avoiding misjudgment. In addition, by setting the number of requests and the waiting time, the duration of connectivity check can be controlled to prevent connection establishment timeout.

[0070] In one optional embodiment, the calling terminal is configured to obtain a candidate pair with connectivity through the following steps: After the calling user triggers the screen sharing signal, a second candidate list is established based on the collected local address and the server reflection address and relay address obtained from the STUN / TURN server. The second candidate list includes host candidates, server reflection candidates and relay candidates sorted by priority from high to low, as well as the priority index of each candidate. Obtain the first candidate list established by the media server; Multiple candidate pairs are formed based on the first candidate list and the second candidate list, and a priority index is calculated for each candidate pair. ICE connectivity checks are performed on each candidate pair in descending order of priority index until a candidate pair with connectivity is obtained.

[0071] Specifically, in conjunction with the above embodiments, the calling terminal will use a similar processing flow as the MS to obtain candidate pairs with connectivity. In specific implementation, the calling terminal launches the SDK and displays a user interface on the screen. After the calling user triggers the screen sharing signal through the user interface, the calling terminal can first establish a second candidate list based on the collected local address of the calling terminal and the server reflection address and relay address obtained from the STS. The established second candidate list includes host candidates, server reflection candidates, and relay candidates sorted in descending order of priority (i.e., the priority of host candidates is higher than that of server reflection candidates, which is higher than that of relay candidates). The second candidate list also includes the priority index corresponding to each candidate. Then, the first candidate established by the MS is obtained from the SIPS. The calling terminal then pairs candidates based on the first candidate list (equivalent to the calling terminal's remote candidate list) and the second candidate list (equivalent to the calling terminal's local candidate list), forming multiple candidate pairs. Each candidate pair includes one candidate from the first candidate list (equivalent to the calling terminal's remote candidate) and one candidate from the second candidate list (equivalent to the calling terminal's local candidate). The calling terminal then calculates the priority index for each candidate pair. Next, the calling terminal performs ICE connectivity checks on each candidate pair sequentially, from highest to lowest priority index, until a connectivity candidate pair is found. This connectivity candidate pair is then designated as an available candidate pair, which is consistent with the available candidate pair found by the MS.

[0072] The present invention provides a 5G new call screen sharing method based on the ICE protocol. After the calling terminal triggers screen sharing, it constructs a second candidate list according to the same rules as the MS, obtains the first candidate list of the MS, forms corresponding candidate pairs, calculates the priority of the candidate pairs, and then performs ICE connectivity checks in descending order of priority to determine the available candidate pairs. This ensures that the ICE connection establishment logic of the calling terminal and the MS is completely symmetrical, thereby ensuring that the available candidate pairs selected by both parties are consistent, reducing connection failures caused by logical differences, and improving the coordination of the establishment of the second video data interaction channel.

[0073] In one optional embodiment, the calling terminal is configured to obtain the first candidate list established by the media server through the following steps: Construct the second SDP Offer information based on the second candidate list, username, password, and room number; The second SDP Offer information is sent to the SIP server, so that the SIP server can parse the received second SDP Offer information to obtain the room number, and determine whether the first SDP Offer information sent by the media server with the same room number already exists in the cache. If so, the second SDP Offer information is constructed based on the first SDP Offer information and returned to the calling terminal. Receive and parse the second SDP Answer information to obtain the first candidate list carried in the second SDP Answer information.

[0074] Specifically, in conjunction with the above embodiments, when the calling terminal obtains the first candidate list established by the MS, it can, after establishing the second candidate list, construct second SDP Offer information based on the information such as each candidate, username, password, and room number contained in the second candidate list, and initiate a request to SIPS based on the second SDP Offer information. Simultaneously, the MS will also adopt a similar processing flow as the calling terminal, constructing first SDP Offer information based on the information such as each candidate, username, password, and room number contained in the first candidate list, and initiating a request to SIPS based on the first SDP Offer information. After receiving the request initiated by the calling terminal, SIPS obtains the information such as each candidate, username, password, and room number carried in the received second SDP Offer information by parsing it, and checks whether the first SDP Offer information sent by the MS already exists in the SIPS cache and whether the first SDP Offer information carries the same room number (i.e., whether the room number sent by the calling terminal is the same as the room number sent by the MS). If so, it constructs second SDP Answer information based on the SDP Offer information sent by the Peer (i.e., the MS) in the cache (i.e., the first SDP Offer information), and sends the second SDP Offer to SIPS. Answer information is returned to the calling terminal. Similarly, after receiving the request initiated by the MS, SIPS obtains the information such as each candidate, username, password, and room number carried in the received first SDP Offer information by parsing the first SDP Offer information. It also checks whether the second SDP Offer information sent by the calling terminal already exists in the SIPS cache and whether the second SDP Offer information carries the same room number (i.e., whether the room number sent by the MS is the same as the room number sent by the calling terminal). If so, it constructs the first SDP Offer information based on the SDP Offer information sent by the Peer (i.e., the calling terminal) in the cache (i.e., the second SDP Offer information) and returns the first SDP Answer information to the MS. The calling terminal receives the second SDP Offer information returned by SIPS and parses the received second SDP Answer information to obtain the first candidate list carried in the first SDP Offer information corresponding to the second SDP Answer information.

[0075] The present invention provides a 5G new call screen sharing method based on the ICE protocol. The calling terminal constructs a second SDP Offer information based on a second candidate list, username, password, and room number. It then matches the first SDP Offer information of the MS through SIPS relay and parses the first candidate list of the MS in the first SDP Offer information. By verifying the uniqueness of the room number, it can ensure that the calling terminal and the MS of the same call exchange candidate lists, thereby improving information security. Furthermore, with the help of SIPS caching and forwarding, it can avoid the loss or tampering of the candidate list, thereby ensuring the integrity of the candidate list.

[0076] In one optional embodiment, the calling terminal obtains the data packet to be shared through the following steps: The system performs real-time capture and format conversion of local screen information to obtain video frame data. The video frame data is subjected to H264 encoding, packet splitting or merging, and FEC encoding to obtain several encoded data packets belonging to the video frame data; wherein, the several encoded data packets include several source data packets belonging to the video frame data and at least one redundant packet; The encoded data packets belonging to the video frame data are numbered to obtain several data packets to be shared belonging to the video frame data.

[0077] Specifically, in conjunction with the above embodiments, when actually performing screen sharing data transmission, the calling terminal can first perform real-time capture and format conversion processing on the local screen information to obtain video frame data (each real-time capture and format conversion process yields one video frame data). Then, the currently obtained video frame data (as the current video frame data) is sequentially processed by H264 encoding, packet splitting or merging, and FEC (Forward Error Correction) encoding to obtain several encoded data packets belonging to the current video frame data. These encoded data packets are actually several source data packets and at least one redundant packet belonging to the current video frame data. Finally, the encoded data packets belonging to the current video frame data are numbered sequentially to obtain several data packets to be shared belonging to the current video frame data.

[0078] It should be noted that the embodiments of the present invention can solve the problem of packet loss that easily occurs in weak network environments by introducing FEC forward redundancy error correction technology. FEC forward error correction technology can adapt to different network environments by setting the number of source data packets and redundant packets. For example, in order to meet the packet loss tolerance rate of 40%, two redundant packets can be generated for every three source data packets. When any two packets are lost, the remaining three packets can be used to recover the three source data packets. However, in practical applications, a balance between bandwidth utilization and video quality should be ensured.

[0079] It should be noted that the embodiments of the present invention can solve the problem of packet out-of-order delivery in weak network environments by numbering data packets. For example, a Sequence byte can be added before the data packet to represent the number of FEC encodings. The Sequence is incremented from 0 and reset to 0 after reaching 255. For example, 3 source data packets, after one FEC encoding, result in 5 data packets to be shared in sequence (i.e., 3 source data packets + 2 redundant packets), with corresponding numbers Sequence=0, 1, 2, 3, and 4. Accordingly, after the MS receives the data packets to be shared from the calling terminal, it can perform subsequent processing based on the numbers of the data packets to be shared.

[0080] The present invention provides a 5G new call screen sharing method based on the ICE protocol. The calling terminal intercepts and converts local screen information, then processes it through H264 encoding, packet splitting / merging, and FEC encoding. The encoded data packets are then numbered to generate data packets to be shared. This method can convert heterogeneous screen information into standardized encoded data packets, adapting to the transmission requirements of the IMS communication network. At the same time, the numbering process provides a basis for subsequent out-of-order correction, which can improve the reliability of data transmission and ensure the quality of screen sharing.

[0081] In one optional embodiment, the calling terminal performs real-time capture and format conversion of local screen information to obtain video frame data, specifically including: Capture local screen information in real time according to preset screenshot parameters to obtain RGB format image data; The RGB format image data is converted to YUV420P format to obtain video frame data.

[0082] Specifically, in conjunction with the above embodiments, when the calling terminal performs real-time capture and format conversion processing on local screen information, it can first capture local screen information in real time according to the preset capture parameters, obtain RGB format image data accordingly, and then convert the obtained RGB format image data into YUV420P format to obtain video frame data (screen information exists in YUV420P format).

[0083] It should be noted that the captured screen information is generally stored as RGB image data. To improve memory utilization and encoding efficiency, it is necessary to convert the RGB format to YUV420P format. According to the ITU-R BT.601 standard, the original RGB-YUV420P conversion formula is as follows: Y=0.299×R+0.587×G+0.114×B, U=-0.168736×R-0.331264×G+0.5×B+128, V=0.5×R-0.418688×G-0.081312×B+128.

[0084] Considering the scenario of real-time screen sharing during 5G calls, video frame data is continuously generated, and format conversion is performed multiple times. Since fixed-point numbers are much more efficient than floating-point numbers, the original conversion formula can be approximated as a fixed-point number. By replacing the denominator with a shift operation, the final RGB-YUV420P conversion formula is obtained as follows: Y=((66×R+129×G+25×B+128)>>8)+16, U=((-38×R-74×G+112×B+128)>>8)+128, V=((112×R-94×G-18×B+128)>>8)+128.

[0085] It should be noted that a caching mechanism can be designed during the format conversion calculation process, so that each R, G, and B value is calculated as a unique value according to the following formula: rgb_key=(R<<16)|(G<<8)|B, The following data structure is designed to store YUV values: typedef struct {uint8_t Y;uint8_t U;uint8_t V;} YUV; Finally, the pointer-type YUV is stored using rgb_key as the key.

[0086] It should be noted that by designing a caching mechanism, multiple calculations can be replaced with negligible query latency. This is especially useful in cases with high resolution and large data volume, as it can reduce the latency of each conversion, thereby greatly improving efficiency.

[0087] The present invention provides a 5G new call screen sharing method based on the ICE protocol. After the calling terminal captures the screen information in RGB format, it converts it into video frame data in YUV420P format, which can adapt to the input requirements of subsequent H264 encoding, reduce the format conversion overhead during encoding, and improve encoding efficiency. By utilizing the chroma subsampling characteristics of YUV420P, the amount of data transmission can be reduced while ensuring image quality.

[0088] In one alternative embodiment, the screenshot parameters include resolution and frame rate.

[0089] Specifically, in conjunction with the above embodiments, the pre-set screenshot parameters mainly include resolution (there is no strict limitation on the accuracy requirements of video resolution) and frame rate. For example, the resolution can be 540×960 and the frame rate can be 30fps, or other suitable parameter values ​​can be set. The embodiments of the present invention do not make specific limitations.

[0090] In one optional embodiment, the calling terminal performs H.264 encoding, packet splitting or merging, and FEC encoding on the video frame data to obtain several encoded data packets belonging to the video frame data, specifically including: The video frame data is input into the H264 encoder, and the video frame data is H264 encoded according to the preset H264 encoding parameters to obtain H264 encoded data belonging to the video frame data. The H264 encoded data belonging to the video frame data is split or combined to obtain several source data packets belonging to the video frame data. According to the preset FEC encoding parameters, several source data packets belonging to the video frame data are subjected to FEC encoding processing to obtain several encoded data packets belonging to the video frame data.

[0091] Specifically, in conjunction with the above embodiments, when the calling terminal performs H264 encoding, packet splitting or merging, and FEC encoding processing on the current video frame data, it can first input the current video frame data into the H264 encoder, perform H264 encoding processing on the current video frame data according to the preset H264 encoding parameters, and obtain the corresponding H264 encoded data belonging to the current video frame data; then, it can perform packet splitting or merging processing on the H264 encoded data belonging to the current video frame data, and obtain several source data packets belonging to the current video frame data accordingly; on this basis, using the FEC encoder, it can perform FEC encoding processing on the several source data packets belonging to the current video frame data according to the preset FEC encoding parameters, and obtain several encoded data packets belonging to the current video frame data accordingly.

[0092] It should be noted that the data channel is based on the UDP protocol. Considering the MTU (Maximum Transmission Unit) size and the IP layer's automatic packet splitting mechanism, custom packet splitting or merging is essential to maximize bandwidth utilization. Furthermore, in shared screen scenarios, H264 video frames all begin with a Start Code (the encoder guarantees that the video data will not be the same as the Start Code). Regardless of whether packets are split or merged, complete video frame data can be recovered. Moreover, video frame data is continuously fed into the encoder, and the encoder's output is also continuous. This allows for packet merging without affecting video quality (low latency).

[0093] The present invention provides a 5G new call screen sharing method based on the ICE protocol. The calling terminal compresses video frame data using H.264 encoding, then generates source data packets by packetizing / merging according to network MTU requirements. Finally, it generates encoded data packets containing redundant packets using FEC encoding. This method can significantly compress the screen data volume through H.264 encoding, improving transmission efficiency. It can also avoid data packet fragmentation through packet fragmentation / merging, adapt to UDP transmission characteristics, and save bandwidth to the maximum extent. Furthermore, the redundant packets generated by FEC encoding can recover the original data in the event of packet loss, ensuring the smoothness of screen sharing.

[0094] In one optional embodiment, the H264 encoding parameters include encoding level and frame rate, and the FEC encoding parameters include the number of source packets and the total number of encoded packets.

[0095] Specifically, in conjunction with the above embodiments, the pre-set H264 encoding parameters mainly include the encoding level and frame rate, and the pre-set FEC encoding parameters mainly include the number of source packets and the total number of encoded packets. For example, the encoding level can be level=3.0, the frame rate can be 30fps, or other suitable parameter values ​​can be set. The embodiments of the present invention do not make specific limitations.

[0096] In one optional embodiment, the H264 encoder uses ABR rate control, and the higher the rate, the fewer redundant packets are set during FEC encoding.

[0097] Specifically, in conjunction with the above embodiments, the H.264 encoder can adopt the ABR (Average Bitrate) bitrate control method, so that the bitrate will vary within a certain range and tend to approach the set expected value as much as possible. For example, the bitrate range corresponding to standard definition is 0~960, and the expected value is 800; the bitrate range corresponding to high definition is 0~1700, and the expected value is 1500 (unit: kbps). Based on the above characteristics, the embodiments of the present invention can be designed to satisfy the following: if the bitrate is large, a smaller number of redundant packets are set during FEC encoding; if the bitrate is small, a larger number of redundant packets are set during FEC encoding.

[0098] The present invention provides a new 5G call screen sharing method based on the ICE protocol. The H264 encoder adopts the ABR bitrate control mode, and the bitrate is negatively correlated with the number of redundant packets in the FEC encoding. It can dynamically balance bandwidth consumption and packet loss resistance. At high bitrates, redundant packets are reduced to save bandwidth, and at low bitrates, redundant packets are increased to improve reliability.

[0099] In one optional embodiment, the H264 encoded data includes IDP frames or P frames, and IDP frames are packetized and P frames are combined.

[0100] Specifically, in conjunction with the above embodiments, in a shared screen scenario, H264 video frames will only be IDR frames (Instantaneous Decoding Refresh) or P frames (Predictive Frames). IDR frames include PPS (Picture Parameter Set), SPS (Sequence Parameter Set), and SEI (Supplemental Enhancement Information). IDR frames are also called keyframes, which the receiver can directly decode and are mostly used for display restoration. P frames are also called non-keyframes, which the receiver needs to rely on other frames for decoding. Based on these characteristics, the embodiments of the present invention can be designed to satisfy the following: For IDR frames, their data volume is large, requiring packet processing, and when the number of packets is too large, grouping for FEC encoding is also necessary; for P frames, their data volume is small, requiring packet merging and FEC encoding.

[0101] The present invention provides a 5G new call screen sharing method based on the ICE protocol. By splitting IDP frames into packets and merging P frames into packets, the transmission method can be optimized for the characteristics of different frame types. Splitting IDP frames can avoid packet loss caused by excessively large single frame data, while merging P frames can reduce the number of data packets, thereby improving the transmission efficiency of different frame types and ensuring the integrity of video frames.

[0102] In one alternative embodiment, the maximum number of redundant packets set for IDP frames is greater than the maximum number of redundant packets set for P frames.

[0103] Specifically, in conjunction with the above embodiments, in addition to limiting the processing methods of IDP frames and P frames, the embodiments of the present invention can also be designed to satisfy the following: the upper limit of the number of redundant packets set in the IDP frame is higher than the upper limit of the number of redundant packets set in the P frame.

[0104] The 5G new call screen sharing method based on the ICE protocol provided in this embodiment of the invention sets the maximum number of redundant packets for IDP frames to be greater than the maximum number of redundant packets for P frames. This can prioritize the transmission reliability of IDP frames, while controlling the number of redundant packets for P frames can avoid excessive bandwidth occupation and balance the transmission resource allocation between IDP frames and P frames.

[0105] In one optional embodiment, the step of performing data recovery processing on the received data packet to be shared, and performing compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate a data packet to be displayed, specifically includes: Each received data packet to be shared is parsed to obtain the number of each data packet to be shared; For several data packets belonging to the same video frame data that are to be shared, out-of-order correction processing is performed based on the number to obtain several data packets to be shared after order correction. FEC decoding and frame reassembly are performed on several data packets to be shared after the order of the video frame data belonging to the same frame data to obtain the H264 encoded data corresponding to the same frame data. Based on the video format standard of the IMS communication network, the H264 encoded data corresponding to the same video frame data is processed for compatibility to generate a data packet to be displayed.

[0106] Specifically, in conjunction with the above embodiments, when the MS performs data recovery processing on the received data packets to be shared and performs compatibility processing on the recovered data according to the video format standard of the IMS communication network, it can first parse each received data packet to be shared to obtain the corresponding number of each data packet to be shared, and identify several data packets belonging to the same video frame (for example, it can be identified according to the frame identifier and other information carried in the data packets to be shared). Then, for the several data packets belonging to the same video frame, it performs out-of-order correction processing based on the parsed numbers to obtain several data packets to be shared after order correction, that is, several data packets to be shared ordered according to the correct number order. Next, it performs FEC decoding processing and frame reassembly processing (i.e., packet splitting / packet merging reverse processing) on ​​the several data packets to be shared after order correction belonging to the same video frame in sequence to obtain the H264 encoded data (i.e., a complete video frame data) corresponding to the same video frame data. Finally, according to the video format standard of the IMS communication network, it performs compatibility processing on the H264 encoded data corresponding to the same video frame data to generate the corresponding data packets to be displayed.

[0107] It should be noted that by performing out-of-order correction processing on the received data packets to be shared, it can be ensured that the MS can complete subsequent processing procedures such as FEC decoding in the order of packets sent by the calling terminal, thereby ensuring the quality of video calls.

[0108] The present invention provides a 5G new call screen sharing method based on the ICE protocol. This method uses MS to parse the number of the data packet to be shared, corrects out-of-order data in the same frame, then performs FEC decoding and frame reassembly to recover H264 encoded data. Finally, it performs compatibility processing according to the IMS format standard to generate the data packet to be displayed. This solves the problem of out-of-order data packets in UDP transmission, recovers lost source data, and converts the processed H264 encoded data into a data format compatible with IMS communication networks. This ensures that the called terminal can correctly parse and display screen content, thereby improving system flexibility and enhancing system scalability.

[0109] In one optional embodiment, the step of performing compatibility processing on the H264 encoded data corresponding to the same video frame data according to the video format standard of the IMS communication network to generate a data packet to be displayed specifically includes: According to the video format standard of the IMS communication network, the H264 encoded data corresponding to the same video frame data is encapsulated with RTP payload to generate a data packet to be displayed.

[0110] Specifically, in conjunction with the above embodiments, when MS performs compatibility processing on the H264 encoded data corresponding to the same video frame data according to the video format standard of the IMS communication network, it can perform RTP (Real-time Transport Protocol) payload encapsulation operation on the H264 encoded data corresponding to the same video frame data according to the video format standard of the IMS communication network, such as referring to the RFC6184 standard, and generate an IMS communication network compatible RTP data packet, which is the data packet to be displayed.

[0111] The present invention provides a new 5G call screen sharing method based on the ICE protocol. By encapsulating H264 encoded data into RTP data packets according to the video format standard of the IMS communication network, the H264 encoded data can be adapted to the RTP protocol of the IMS communication network and transmitted directly in the IMS communication network. Furthermore, by using the timestamp and sequence number mechanism of RTP, the timing and integrity of video frame data can be guaranteed, thereby further improving the real-time performance of screen sharing.

[0112] It should be noted that after receiving the data packet to be displayed (i.e., RTP data packet) sent by the MS, the called terminal can first obtain the H264 encoded data by parsing, and then obtain the YUV420P format video frame data by H264 decoding. After that, the decoded YUV420P format video frame data is converted into RGB format image data, which can restore the screen information shared by the calling terminal and display it on the screen of the called terminal in real time, thus completing real-time screen sharing.

[0113] See Figure 4 The diagram shown is a data interaction flowchart based on the ICE protocol provided in an embodiment of the present invention. Figure 4 This mainly demonstrates the process of establishing a second video data interaction channel between the calling terminal and the MS based on the ICE protocol, combined with... Figure 4 The specific establishment process is as follows, similar to the above embodiments: MS: Collect local addresses, obtain server reflection addresses and relay addresses from STS through the address request-response mechanism, form host candidates, server reflection candidates and relay candidates respectively, and quantify and sort the priority indicators of each candidate in descending order to form a first candidate list; construct the first SDP Offer information based on the information of each candidate, username, password and room number contained in the first candidate list, and send it to SIPS; SIPS: Receive the first SDP Offer information sent by the MS, parse it to obtain the room number, and check whether the calling terminal has sent the second SDP Offer information and whether the second SDP Offer information carries the same room number. If so, construct the first SDP Answer information based on the second SDP Offer information sent by the calling terminal and return it to the MS. Calling terminal: Collects local addresses, obtains server reflection addresses and trunk addresses from STS through address request-request response mechanism, and forms host candidates, server reflection candidates and trunk candidates respectively. It then quantifies and sorts the priority indicators of each candidate in descending order to form a second candidate list. Based on the information of each candidate, username, password and room number contained in the second candidate list, it constructs a second SDP Offer information and sends it to SIPS. SIPS: Receive the second SDP Offer information sent by the calling terminal, parse it to obtain the room number, and check whether the MS has sent the first SDP Offer information and whether the first SDP Offer information carries the same room number. If so, construct the second SDP Answer information based on the first SDP Offer information sent by the MS and return it to the calling terminal. MS: Receives the first SDP Answer information returned by SIPS, parses it to obtain the second candidate list, forms multiple candidate pairs based on the first and second candidate lists, calculates the priority index of each candidate pair, and performs ICE connectivity checks on each candidate pair in descending order of priority index through the STUNBinding Request-STUN Binding Response mechanism until a candidate pair with connectivity is found, and takes the found candidate pair with connectivity as the available candidate pair; The calling terminal receives the second SDP Answer information returned by SIPS, parses it to obtain the first candidate list, forms multiple candidate pairs based on the first and second candidate lists, calculates the priority index of each candidate pair, and performs ICE connectivity checks on each candidate pair in descending order of priority index through the STUN Binding Request-STUN Binding Response mechanism until a candidate pair with connectivity is found. The found candidate pair with connectivity is then designated as an available candidate pair. The available candidate pair found by the calling terminal is the same as the available candidate pair found by the MS. Finally, a second video data interaction channel from the calling terminal to the MS can be established based on the available candidate pairs.

[0114] Furthermore, Figure 4 It also showed the process of the calling terminal sharing its screen with the called terminal in real time via MS, combined with... Figure 4 The specific sharing process is as follows, similar to the above embodiments: The calling terminal: captures local screen information in real time via screenshot, converts the format, and sends it to the H.264 encoder for H.264 encoding; it then performs packet splitting or merging of the H.264 encoded data, followed by FEC encoding; it numbers the FEC-encoded data packets and sends the numbered data packets to be shared to the MS (i.e., MS) through the second video data interaction channel. Figure 4 The “Data” shown refers to the transmission process from the calling terminal to the MS. MS: Receives the data packet to be shared from the calling terminal, obtains its number through parsing, and performs out-of-order correction based on the number; performs FEC decoding and frame reassembly on the corrected data packet to obtain H.264 encoded data; performs RTP payload encapsulation on the H.264 encoded data, and sends the generated data packet to be displayed to the called terminal (i.e., ...) through the first video data interaction channel. Figure 4 The “Data” shown refers to the transmission process from the MS to the called terminal. The called terminal receives the data packet to be displayed sent by the MS, obtains the H264 encoded data through RTP parsing, performs H264 decoding on the H264 encoded data, restores the decoded video frame data to the screen information shared by the calling terminal through format conversion, and displays it in real time, thereby completing real-time screen sharing.

[0115] It should be noted that, in combination Figure 3 and Figure 4 As shown, after receiving the StopSession(all) request from the AS, the MS, in addition to closing the audio and video data interaction channels with the calling and called terminals, will also begin to release relevant resources. Specifically: the MS sends a BYE signaling to the SIPS; after receiving the BYE signaling from the MS, the SIPS completes the resource release and returns a 200 OK signaling to the MS in response to the BYE signaling; after receiving the 200 OK signaling returned by the SIPS, the MS completes the resource release and returns status information to the AS; after receiving the status information returned by the MS, the AS completes the resource release; at this point, the call between the calling and called parties ends, and the relevant resource release is completed.

[0116] In summary, the 5G new call screen sharing method based on the ICE protocol provided by this invention does not require upgrading existing media network elements and terminal hardware, thus avoiding additional equipment and implementation costs; it enables data interaction between the terminal's intranet and the IMS communication network without requiring the terminal to download a mini-program during a call, achieving real-time screen sharing between the caller and the called party; considering the special characteristics of screen sharing scenarios, it also provides targeted solutions for packet loss and out-of-order packet problems in weak network environments, maximizing bandwidth utilization and greatly enhancing video effects and user experience; it can quickly upgrade call functions at a low cost, and all basic call functions are retained during the upgrade process, ensuring the user's basic communication needs.

[0117] In addition, the embodiments of the present invention also have the following beneficial effects: (1) It has stability. Except for the Internet channel between the calling terminal and the MS, the other channels use a very stable and reliable IMS communication network, which can guarantee basic voice functions even in weak network conditions; (2) It has scalability. The Internet channel is established between the calling terminal and the MS, which builds a communication bridge from the Internet to the IMS communication network, laying a communication foundation for subsequent scenario innovation, and also providing new directions for technological innovation in the field of voice communication; (3) It has flexibility. The real-time audio and video communication technology of the Internet has matured, such as FEC redundancy error correction, AQR retransmission (Automatic Repeat-reQuest), packet out-of-order processing and other technologies for weak network countermeasures, but they have not been applied to traditional operator calls. In the embodiments of the present invention, the calling SDK and the mediator MS can easily introduce these technologies without affecting the original media network element architecture, and can be continuously iterated and updated without upgrading the terminal hardware.

[0118] This invention also provides a 5G new call screen sharing device based on the ICE protocol, used to implement the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments. See [link to relevant documentation]. Figure 5 The diagram shown is a structural block diagram of a 5G new call screen sharing device based on the ICE protocol according to an embodiment of the present invention. The device is disposed in a media server and includes: The first video channel establishment module 11 is used to receive a video session creation request sent by the application server and establish a first video data interaction channel from the media server to the called terminal; wherein, the video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal and the audio call between the calling terminal and the called terminal are established. The second video channel establishment module 12 is used to establish a second video data interaction channel from the calling terminal to the media server based on the ICE protocol. The data packet receiving module 13 is used to receive the data packet to be shared sent by the calling terminal through the second video data interaction channel; wherein the data packet to be shared is obtained by the calling terminal after processing the local screen information and then sent to the media server; The data packet generation module 14 is used to perform data recovery processing on the received data packet to be shared, and to perform compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate the data packet to be displayed. The data packet to be displayed sending module 15 is used to send the generated data packet to be displayed to the called terminal through the first video data interaction channel, so that the called terminal can display the screen information of the calling terminal in real time according to the received data packet to be displayed.

[0119] It should be noted that the 5G new call screen sharing device based on the ICE protocol provided in this embodiment of the invention can realize all the processes of the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments. The functions and technical effects of each module and unit in the device are the same as the functions and technical effects of the 5G new call screen sharing method based on the ICE protocol described in the above embodiments, and will not be repeated here.

[0120] This invention also provides a computer-readable storage medium including a stored computer program, which, when running, controls the device where the computer-readable storage medium is located to execute the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments.

[0121] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments.

[0122] This invention also provides a media server, see [link to relevant documentation]. Figure 6 The diagram shown is a structural block diagram of a media server provided in an embodiment of the present invention. The media server includes a processor 10, a memory 20, and a computer program stored in the memory 20 and configured to be executed by the processor 10. When the processor 10 executes the computer program, it implements the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments.

[0123] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory 20 and executed by the processor 10 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the media server.

[0124] It should be noted that the aforementioned media server may include, but is not limited to, processors and memory, as will be understood by those skilled in the art. Figure 6 The block diagram shown is merely an example of the structure of the media server described above and does not constitute a limitation on the structure of the media server. The media server may include more or fewer components than shown, or combine certain components, or use different components.

[0125] Combination Figure 2 As shown, this embodiment of the invention also provides a 5G new call screen sharing system based on the ICE protocol, including a calling terminal, a called terminal, an application server, and a media server. The media server is used to cooperate with the calling terminal, the called terminal, and the application server to implement the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments.

[0126] It should be noted that the 5G new call screen sharing system based on the ICE protocol provided in this embodiment of the invention can realize all the processes of the 5G new call screen sharing method based on the ICE protocol described in any of the above embodiments. The execution processes and technical effects of the calling terminal, called terminal, application server and media server in the system are the same as those of the calling terminal, called terminal, application server and media server involved in the 5G new call screen sharing method based on the ICE protocol described in the above embodiments, and will not be repeated here.

[0127] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A new 5G call screen sharing method based on the ICE protocol, characterized in that, Applied to media servers, including: The application server receives a video session creation request and establishes a first video data interaction channel between the media server and the called terminal. The video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal, as well as the audio call between the calling terminal and the called terminal, are established. Based on the ICE protocol, a second video data interaction channel is established between the calling terminal and the media server; The second video data interaction channel receives a data packet to be shared sent by the calling terminal; wherein the data packet to be shared is obtained by the calling terminal after processing its local screen information and then sent to the media server. The received data packets to be shared are processed for data recovery, and the recovered data is processed for compatibility according to the video format standard of the IMS communication network to generate data packets to be displayed. The generated data packet to be displayed is sent to the called terminal through the first video data interaction channel, so that the called terminal can display the screen information of the calling terminal in real time according to the received data packet to be displayed.

2. The 5G new call screen sharing method based on the ICE protocol as described in claim 1, characterized in that, The application server is used to interact with the calling terminal, the called terminal, and the media server to establish audio data interaction channels between the media server and the calling terminal, and between the media server and the called terminal; it is also used to receive and transmit signaling between the calling terminal and the called terminal through a session description protocol to establish an audio call between the calling terminal and the called terminal.

3. The 5G new call screen sharing method based on the ICE protocol as described in claim 1, characterized in that, The establishment of a second video data interaction channel from the calling terminal to the media server based on the ICE protocol specifically includes: Based on the collected local addresses and the server reflection addresses and relay addresses obtained from the STUN / TURN server, a first candidate list is established; wherein, the first candidate list includes host candidates, server reflection candidates and relay candidates sorted by priority from high to low, as well as the priority index of each candidate. Obtain a second candidate list established by the calling terminal; wherein the second candidate list is established by the calling terminal based on the collected local address and the server reflection address and trunk address obtained from the STUN / TURN server, using the same sorting rules as the first candidate list; Multiple candidate pairs are formed based on the first candidate list and the second candidate list, and a priority index is calculated for each candidate pair; wherein each candidate pair includes one candidate from the first candidate list and one candidate from the second candidate list; ICE connectivity checks are performed on each candidate pair in descending order of priority index until a candidate pair with connectivity is obtained, and the obtained candidate pair with connectivity is taken as a usable candidate pair; wherein, the usable candidate pair is the same as the candidate pair with connectivity obtained by the calling terminal based on the first candidate list and the second candidate list; A second video data interaction channel is established between the calling terminal and the media server based on the available candidate pairs.

4. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, The step of establishing a first candidate list based on the collected local addresses and the server reflection addresses and relay addresses obtained from the STUN / TURN server specifically includes: Collect local addresses and send requests to the STUN / TURN server to obtain the server reflection address and relay address; The host candidates corresponding to the local address, the server reflection candidates corresponding to the server reflection address, and the relay candidates corresponding to the relay address are sorted in descending order of priority index to form the first candidate list.

5. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, The acquisition of the second candidate list established by the calling terminal specifically includes: Construct the first SDP Offer information based on the first candidate list, username, password, and room number; The first SDP Offer information is sent to the SIP server, which parses the received SDP Offer information to obtain the room number and checks whether the second SDP Offer information sent by the calling terminal with the same room number already exists in the cache. If so, the first SDP Answer information is constructed based on the second SDP Offer information and returned to the media server. The second SDP Offer information is constructed by the calling terminal based on the established second candidate list, username, password and room number and then sent to the SIP server. Receive and parse the first SDP Answer information to obtain the second candidate list carried in the first SDP Answer information.

6. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, The step of forming multiple candidate pairs based on the first candidate list and the second candidate list specifically includes: According to the order of each candidate in the first candidate list and the order of each candidate in the second candidate list, two candidates in the first candidate list and the second candidate list that are in the same order position are paired up to form multiple candidate pairs.

7. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, Each candidate's priority index is a preset quantitative value, and the higher the preset quantitative value, the higher the priority of the candidate.

8. The 5G new call screen sharing method based on the ICE protocol as described in claim 7, characterized in that, The priority index for each candidate pair is the product of the preset quantization values ​​of the two candidates contained in the candidate pair.

9. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, In each candidate pair, the candidates in the first candidate list are designated as local candidates, and the candidates in the second candidate list are designated as remote candidates. The step of performing ICE connectivity checks on each candidate pair in descending order of priority index specifically includes: Following the traversal order of priority indicators from largest to smallest, for each candidate pair, a STUN binding request is sent from the local candidate to the remote candidate. If no STUN binding response is received from the remote candidate after multiple requests within a preset time period, the corresponding candidate pair is determined to be unconnected; otherwise, the corresponding candidate pair is determined to be connected.

10. The 5G new call screen sharing method based on the ICE protocol as described in claim 3, characterized in that, The calling terminal is used to obtain candidate pairs with connectivity through the following steps: After the calling user triggers the screen sharing signal, a second candidate list is established based on the collected local address and the server reflection address and relay address obtained from the STUN / TURN server. The second candidate list includes host candidates, server reflection candidates and relay candidates sorted by priority from high to low, as well as the priority index of each candidate. Obtain the first candidate list established by the media server; Multiple candidate pairs are formed based on the first candidate list and the second candidate list, and a priority index is calculated for each candidate pair. ICE connectivity checks are performed on each candidate pair in descending order of priority index until a candidate pair with connectivity is obtained.

11. The 5G new call screen sharing method based on the ICE protocol as described in claim 10, characterized in that, The calling terminal is used to obtain the first candidate list established by the media server through the following steps: Construct the second SDP Offer information based on the second candidate list, username, password, and room number; The second SDP Offer information is sent to the SIP server, so that the SIP server can parse the received second SDP Offer information to obtain the room number, and determine whether the first SDP Offer information sent by the media server with the same room number already exists in the cache. If so, the second SDP Offer information is constructed based on the first SDP Offer information and returned to the calling terminal. Receive and parse the second SDP Answer information to obtain the first candidate list carried in the second SDP Answer information.

12. The 5G new call screen sharing method based on the ICE protocol as described in claim 1, characterized in that, The calling terminal obtains the data packet to be shared through the following steps: The system performs real-time capture and format conversion of local screen information to obtain video frame data. The video frame data is subjected to H264 encoding, packet splitting or merging, and FEC encoding to obtain several encoded data packets belonging to the video frame data; wherein, the several encoded data packets include several source data packets belonging to the video frame data and at least one redundant packet; The encoded data packets belonging to the video frame data are numbered to obtain several data packets to be shared belonging to the video frame data.

13. The 5G new call screen sharing method based on the ICE protocol as described in claim 12, characterized in that, The calling terminal performs real-time capture and format conversion of local screen information to obtain video frame data, specifically including: Capture local screen information in real time according to preset screenshot parameters to obtain RGB format image data; The RGB format image data is converted to YUV420P format to obtain video frame data.

14. The 5G new call screen sharing method based on the ICE protocol as described in claim 13, characterized in that, The screenshot parameters include resolution and frame rate.

15. The 5G new call screen sharing method based on the ICE protocol as described in claim 12, characterized in that, The calling terminal performs H.264 encoding, packet splitting or merging, and FEC encoding on the video frame data to obtain several encoded data packets belonging to the video frame data, specifically including: The video frame data is input into the H264 encoder, and the video frame data is H264 encoded according to the preset H264 encoding parameters to obtain H264 encoded data belonging to the video frame data. The H264 encoded data belonging to the video frame data is split or combined to obtain several source data packets belonging to the video frame data. According to the preset FEC encoding parameters, several source data packets belonging to the video frame data are subjected to FEC encoding processing to obtain several encoded data packets belonging to the video frame data.

16. The 5G new call screen sharing method based on the ICE protocol as described in claim 15, characterized in that, The H264 encoding parameters include encoding level and frame rate, and the FEC encoding parameters include the number of source packets and the total number of encoded packets.

17. The 5G new call screen sharing method based on the ICE protocol as described in claim 15, characterized in that, The H264 encoder uses ABR rate control, and the higher the rate, the fewer redundant packets are set during FEC encoding.

18. The 5G new call screen sharing method based on the ICE protocol as described in claim 15, characterized in that, The H264 encoded data includes IDP frames or P frames, and IDP frames are processed by packet segmentation, while P frames are processed by packet merging.

19. The 5G new call screen sharing method based on the ICE protocol as described in claim 18, characterized in that, The maximum number of redundant packets set for IDP frames is greater than the maximum number of redundant packets set for P frames.

20. The 5G new call screen sharing method based on the ICE protocol as described in claim 15, characterized in that, The process of recovering the received data packets to be shared and performing compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate data packets to be displayed specifically includes: Each received data packet to be shared is parsed to obtain the number of each data packet to be shared; For several data packets belonging to the same video frame data that are to be shared, out-of-order correction processing is performed based on the number to obtain several data packets to be shared after order correction. FEC decoding and frame reassembly are performed on several data packets to be shared after the order of the video frame data belonging to the same frame data to obtain the H264 encoded data corresponding to the same frame data. Based on the video format standard of the IMS communication network, the H264 encoded data corresponding to the same video frame data is processed for compatibility to generate a data packet to be displayed.

21. The 5G new call screen sharing method based on the ICE protocol as described in claim 20, characterized in that, The process of performing compatibility processing on the H264 encoded data corresponding to the same video frame according to the video format standard of the IMS communication network to generate a data packet to be displayed specifically includes: According to the video format standard of the IMS communication network, the H264 encoded data corresponding to the same video frame data is encapsulated with RTP payload to generate a data packet to be displayed.

22. A 5G new call screen sharing device based on the ICE protocol, characterized in that, The settings in the media server include: The first video channel establishment module is used to receive a video session creation request sent by the application server and establish a first video data interaction channel from the media server to the called terminal. The video session creation request is sent by the application server to the media server after the audio data interaction channel between the media server and the calling terminal and the called terminal, as well as the audio call between the calling terminal and the called terminal, are established. The second video channel establishment module is used to establish a second video data interaction channel between the calling terminal and the media server based on the ICE protocol. The data packet receiving module is used to receive the data packet to be shared sent by the calling terminal through the second video data interaction channel; wherein the data packet to be shared is obtained by the calling terminal after processing the local screen information and then sent to the media server; The data packet generation module is used to perform data recovery processing on the received data packets to be shared, and to perform compatibility processing on the recovered data according to the video format standard of the IMS communication network to generate the data packets to be displayed. The data packet to be displayed module is used to send the generated data packet to be displayed to the called terminal through the first video data interaction channel, so that the called terminal can display the screen information of the calling terminal in real time according to the received data packet to be displayed.

23. A computer-readable storage medium, characterized in that, The device includes a stored computer program that, when executed, controls the device containing the computer-readable storage medium to perform the 5G new call screen sharing method based on the ICE protocol as described in any one of claims 1 to 21.

24. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the 5G new call screen sharing method based on the ICE protocol as described in any one of claims 1 to 21.

25. A media server, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the 5G new call screen sharing method based on the ICE protocol as described in any one of claims 1 to 21.

26. A new 5G call screen sharing system based on the ICE protocol, characterized in that, It includes a calling terminal, a called terminal, an application server, and a media server. The media server is used to cooperate with the calling terminal, the called terminal, and the application server to implement the 5G new call screen sharing method based on the ICE protocol as described in any one of claims 1 to 21.

Images