Media stream processing method and device, related equipment, storage medium and computer program product

Abstract

The invention discloses a media stream processing method and device, related equipment, a storage medium and a computer program product. The method is applied to a service platform, and comprises the following steps: receiving a first signaling sent by a first function; the first signaling comprises a first request, and the first request represents a remote assistance request initiated by a first terminal; obtaining a first media stream in response to the first request; the first media stream is provided by the first terminal; generating a pull stream address, and sending the pull stream address to a second function, so that the second function obtains the first media stream based on the pull stream address, and edits the first media stream to obtain a second media stream; the second function is set at a personal computer (PC) end of a second terminal, and the first terminal and the second terminal establish a call service connection.

Description

Technical Field

[0001] The present application relates to the technical field of device remote control, and in particular to a media stream processing method, apparatus, related equipment, storage medium and computer program product. Background Art

[0002] Currently, when a user of a local device encounters a problem during a call service, the user can connect to the local device through network remote assistance technology to achieve remote control of the local device.

[0003] In the related technology, the implementation method of remote assistance needs to rely on the audio and video call application (APP) on the terminal side. On the one hand, the local user and the opposite user must download the corresponding APP to use the remote assistance function, which increases the cost of use. Moreover, if it is in a remote customer service scenario, this will undoubtedly increase the user's communication cost and bring inconvenience to the user. At the same time, it makes it difficult for many users who are not proficient in using terminals such as mobile phones to use the remote assistance service. On the other hand, this type of APP uses the mobile phone side for rendering and processing, and has certain requirements on the performance of the mobile phone, making it difficult for mobile phones with poor performance to use the remote assistance service smoothly. Summary of the invention

[0004] In order to solve the technical problems existing in the related technologies, the embodiments of the present application provide a media stream processing method, apparatus, related equipment, storage medium and computer program product.

[0005] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a media stream processing method, which is applied to a service platform, and the method includes:

[0007] receiving a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0008] In response to the first request, obtaining a first media stream; the first media stream is provided by the first terminal;

[0009] Generate a streaming address and send the streaming address to a second function, so that the second function obtains the first media stream based on the streaming address and edits the first media stream to obtain a second media stream; the second function is set on the personal computer (PC) side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0010] In a second aspect, an embodiment of the present application further provides another media stream processing method, which is applied to the second function, and the method includes:

[0011] Receive a stream pull address sent by a service platform; the stream pull address is generated by the service platform;

[0012] A first media stream is obtained based on the stream pull address, and the first media stream is edited to obtain a second media stream; the first media stream is provided by a first terminal, and the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0013] The second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0014] In a third aspect, an embodiment of the present application provides a media stream processing device, which is applied to a service platform, and the device includes:

[0015] A first receiving unit, configured to receive a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0016] A first acquiring unit, configured to acquire a first media stream in response to the first request; the first media stream is provided by the first terminal;

[0017] A generating unit, used to generate a stream pulling address;

[0018] The first sending unit is used to send the stream pulling address to the second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain the second media stream; the second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0019] In a fourth aspect, an embodiment of the present application further provides another media stream processing device, applied to the second function, the device comprising:

[0020] A second receiving unit is used to receive a stream pulling address sent by a service platform; the stream pulling address is generated by the service platform;

[0021] A second acquisition unit, configured to acquire a first media stream based on the stream pull address;

[0022] an editing unit, configured to edit the first media stream to obtain a second media stream; the first media stream is provided by a first terminal, the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0023] The second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0024] In a fifth aspect, an embodiment of the present application provides a service platform, including: a first processor and a first memory for storing a computer program that can be run on the first processor;

[0025] Wherein, the first processor is used to execute the steps of the media stream processing method on the service platform side described in the embodiment of the present application when running the computer program.

[0026] In a sixth aspect, an embodiment of the present application provides a second function, including: a second processor and a second memory for storing a computer program that can be run on the second processor;

[0027] Wherein, when the second processor is used to run the computer program, it executes the steps of the media stream processing method on the second functional side described in the embodiment of the present application.

[0028] In the seventh aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the media stream processing method on the service platform side described in the embodiment of the present application are implemented, or the steps of the media stream processing method on the second function side described in the embodiment of the present application are implemented.

[0029] In an eighth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the media stream processing method on the service platform side described in the embodiment of the present application, or implements the steps of the media stream processing method on the second function side described in the embodiment of the present application.

[0030] The media stream processing method, apparatus, related equipment, storage medium and computer program product provided in the embodiments of the present application are as follows: a service platform receives a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by a first terminal; in response to the first request, a first media stream is obtained; the first media stream is provided by the first terminal; a stream pulling address is generated, and the stream pulling address is sent to a second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain a second media stream; the second function is set on the PC end of the second terminal, and a call service connection is established between the first terminal and the second terminal. By adopting the technical solution of the embodiment of the present application, the second function is set on the PC side of the second terminal, so that the first media stream provided by the first terminal is forwarded to the PC side of the second terminal, and then the first media stream is edited by the PC side of the second terminal to obtain the second media stream. In this way, there is no need to install any APP, and only the call service needs to be used to realize the remote assistance service function, which lowers the user's usage threshold, avoids the use of APP to provide remote assistance services, and reduces the use cost of the first terminal, thereby providing remote assistance services to users of the first terminal more conveniently and quickly; at the same time, by setting the stream pulling editing function on the PC side of the second terminal, the performance requirements for the second terminal are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 1 ;

[0032] Figure 2 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 2 ;

[0033] Figure 3 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 3 ;

[0034] Figure 4 A schematic diagram of a first terminal and a remote assistance function module according to an embodiment of the present application;

[0035] Figure 5 A flowchart of the remote assistance service process according to an embodiment of the present application;

[0036] Figure 6 This is a schematic diagram of the media stream push process of an embodiment of the present application;

[0037] Figure 7 The structure of the media stream processing device of the present invention is shown in FIG. Figure 1 ;

[0038] Figure 8 The structure of the media stream processing device of the present invention is shown in FIG. Figure 2 ;

[0039] Fig. 9 A schematic diagram of the hardware structure of the service platform of the embodiment of the present application;

[0040] Fig.10 A schematic diagram of the hardware structure of the second function of an embodiment of the present application;

[0041] Fig.11 A schematic diagram of the structure of a media stream processing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] The present application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0044] At present, remote assistance for video calls is generally implemented based on the audio and video call APP on the terminal side. Specifically, the audio and video streams are collected through the audio and video call APP on the local device side, and the collected audio and video streams are pushed to the other side. The audio and video streams are received through the audio and video call APP on the other side, and the relevant audio and video stream editing is performed. The edited audio and video streams are then pushed back to the local device side, thereby completing the business process of remote assistance for video calls.

[0045] Among them, in the business process of remote assistance, the following methods can be used to edit audio and video streams: direct overlay, frozen screen overlay, whiteboard assistance, and augmented reality (AR) overlay. Taking video stream as an example, direct overlay is to add various annotations to the video stream, which will cause the annotations to move with the displacement of the video stream and cannot be accurately located; frozen screen overlay is to freeze the video stream screen and annotate under a single video frame, and the editing is relatively stable and reliable; whiteboard assistance is to complete the annotation of the video stream on a whiteboard; AR overlay is to collect the feature information of the object in the video content and anchor the marking signal on the object to be marked.

[0046] It can be seen that the advantage of the video call remote assistance implementation method of the related technology is that it is relatively easy to implement, and the audio and video streams of the other party can be taken over on the mobile phone. It can also easily realize a variety of video call functions, etc. However, the existing problems are also obvious: First, the video call remote assistance implementation method of the related technology needs to rely on the audio and video call APP on the terminal side. On the one hand, local users and peer users must download the corresponding APP to use the remote assistance function, which increases the user's use cost. Moreover, if in a remote customer service scenario, this will undoubtedly increase the user's communication cost and bring inconvenience to the user. At the same time, it makes it difficult for many users who are not proficient in using terminals such as mobile phones, especially the elderly and children, to use the remote assistance service; on the other hand, this type of APP uses the mobile phone side for rendering and processing, and has certain requirements on the performance of the mobile phone, making it difficult for mobile phones with poor performance to use the remote assistance service smoothly.

[0047] Based on this, an embodiment of the present application proposes a media stream processing method. In various embodiments of the present application, by setting the second function on the PC side of the second terminal, the first media stream provided by the first terminal is forwarded to the PC side of the second terminal, and then the first media stream is edited by the PC side of the second terminal to obtain the second media stream. In this way, there is no need to install any APP, and only the call service needs to be used to realize the remote assistance service function, which lowers the user's usage threshold, avoids the use of APP to provide remote assistance services, and reduces the use cost of the first terminal, thereby providing remote assistance services to users of the first terminal more conveniently and quickly; at the same time, by setting the stream pulling editing function on the PC side of the second terminal, the performance requirements for the second terminal are reduced.

[0048] The present application embodiment provides a media stream processing method, which is applied to a service platform. Figure 1 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 1 ;like Figure 1 As shown, the method includes:

[0049] Step 101: Receive a first signaling sent by a first function; the first signaling includes a first request.

[0050] In an embodiment of the present application, the first request represents a remote assistance request initiated by the first terminal, and the first terminal can be understood as the calling party, that is, the party being assisted (referred to as the assisted party). Here, the first function represents a Session Initiation Protocol (SIP) signaling server, and the service platform represents a media capability platform, that is, the media capability platform receives a first signaling sent by the SIP signaling server, and the first signaling includes a remote assistance request initiated by the first terminal.

[0051] Here, the first function is the SIP signaling server, which can be developed based on the C++ language, and the SIP signaling server is mainly responsible for the interaction of SIP signaling and Hypertext Transfer Protocol (HTTP) requests. The business platform is the media capability platform, which can also be developed based on the C++ language, and the media capability platform is mainly responsible for receiving and pushing media streams and providing streaming addresses.

[0052] In actual application, when a native call connection is established between the first terminal and the second terminal, the video call service is triggered to start during the native call, such as the video call service in a new call of the fifth generation mobile communication technology (5G, 5th Generation Mobile Communication Technology), and after the video call connection is established, the first function sends a first signaling to the service platform to initiate a remote assistance request to the service platform through the first signaling.

[0053] Based on this, in one embodiment, the receiving of the first signaling sent by the first function includes: receiving the first signaling sent by the first function during a video call between the first terminal and the second terminal; wherein the video call is triggered by a native call between the first terminal and the second terminal.

[0054] Here, the second terminal can be understood as the called party, that is, the party providing assistance (referred to as the assisting party). In an embodiment of the present application, the first terminal first initiates a native call request to the second terminal. After the second terminal agrees to the request, a native call connection is established between the first terminal and the second terminal. Next, the user of the first terminal dials a video call through the native call on the mobile phone, that is, the video call is triggered based on the native call between the first terminal and the second terminal, and remote assistance service is implemented during the video call. Exemplarily, remote assistance service is implemented in the new 5G call scenario without installing any APP, which reduces the user's usage threshold.

[0055] Here, in actual application, the first signaling sent by the first function to the service platform may be a SIP signaling, and the SIP signaling includes a remote assistance request, that is, the first function sends a remote assistance request to the service platform in the form of a SIP signaling.

[0056] It should be noted that when the user of the first terminal dials a video call through the native call on the mobile phone, it will trigger the core network device to send SIP signaling to the first function, so that the first function forwards the SIP signaling to the service platform. The core network device here may include an IP Multimedia Subsystem (IMS). It can be seen that the first function, namely the SIP signaling server, is a signaling transfer station. This is because the core network device uses SIP signaling communication, and the service platform uses HTTP communication. Therefore, the first function, namely the SIP signaling server, is required to be a signaling transfer station to realize the interaction of SIP signaling and HTTP requests.

[0057] Step 102: In response to the first request, obtain a first media stream.

[0058] In the embodiment of the present application, the first media stream is provided by the first terminal, wherein the first media stream includes an audio stream and a video stream provided by the first terminal, that is, an audio and video stream provided by the first terminal.

[0059] In actual application, the service platform may obtain the first media stream based on the first information sent by the first function.

[0060] Based on this, in one embodiment, obtaining the first media stream in response to the first request includes: determining whether the first information sent by the first function is received in response to the first request; the first information represents a notification message notifying the service platform to receive the first media stream; and obtaining the first media stream when it is determined that the first information sent by the first function is received.

[0061] Here, in one embodiment, the determining whether the first information sent by the first function is received includes: when the first function negotiates successfully with the service platform regarding the first signaling, determining that the first information sent by the first function is received.

[0062] In actual application, after receiving the first signaling, the service platform can perform signaling negotiation with the first function regarding the first signaling. When the first function and the service platform successfully negotiate the first signaling, the first function will inform the service platform to prepare to receive the first media stream pushed by the core network device. That is, the first function will send the first information to the service platform, and the first information represents a notification message notifying the service platform to receive the first media stream. At this time, the service platform starts to prepare to receive the first media stream.

[0063] Here, in one embodiment, obtaining the first media stream when it is determined that the first information sent by the first function is received includes: obtaining the first media stream pushed by the core network device when it is determined that the first information sent by the first function is received.

[0064] In one embodiment, after the service platform acquires the first media stream, the method further includes: sending a confirmation message for acquiring the first media stream to the first function.

[0065] In actual application, while obtaining the first media stream provided by the first terminal, i.e., the calling party, pushed by the core network device, the service platform may also obtain the media stream provided by the second terminal, i.e., the called party, pushed by the core network device. However, during the remote assistance service, the calling party only needs to obtain the media stream provided by the assisted party and does not need to obtain the media stream provided by the called party. Therefore, after receiving the media stream provided by the called party, the service platform will discard the media stream provided by the called party.

[0066] Based on this, in one embodiment, the method further includes: when acquiring the first media stream, acquiring a third media stream; and when it is detected that the third media stream is provided by the second terminal, discarding the third media stream.

[0067] In the embodiment of the present application, the third media stream represents a media stream provided by the second terminal, ie, the called party.

[0068] Step 103: Generate a stream pulling address, and send the stream pulling address to the second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain a second media stream.

[0069] In the embodiment of the present application, the second function is set on the PC side of the second terminal, wherein the second function represents a remote assistance function module, and the PC side of the second terminal includes but is not limited to a desktop computer, a laptop computer, etc. Here, a call service connection is established between the first terminal and the second terminal, that is, a native call connection is established between the mobile phone side of the first terminal and the mobile phone side of the second terminal.

[0070] Here, the second function, namely the remote assistance function module, is set on the second terminal, namely the PC of the assisting party, and can be developed based on Python language, ffmpeg-Python library and pyqt5 as the core module for realizing the remote assistance service function, and the remote assistance function module is mainly responsible for the editing function of the media stream.

[0071] In actual application, the business platform carries the stream pull address in the HTTP request and sends the stream pull address to the second function through the HTTP request.

[0072] Based on this, in one embodiment, sending the stream pull address to the second function includes:

[0073] Sending a second request to the second function; the second request carries the stream pulling address;

[0074] The sending a second request to the second function includes:

[0075] A second request is sent to the second function via a first communication connection; the first communication connection represents an HTTP connection.

[0076] Here, since the service platform and the second function use HTTP communication, when the service platform sends the streaming address to the second function, it needs to send the streaming address to the second function through HTTP communication, that is, an HTTP connection established between the service platform and the second function. The streaming address can be carried in the second request, which represents an HTTP request.

[0077] In actual application, after acquiring the first media stream and editing the first media stream to obtain the second media stream, the second function can send the second media stream to the service platform, so as to push the second media stream back to the core network device through the service platform.

[0078] Based on this, in one embodiment, after sending the stream pull address to the second function so that the second function obtains the first media stream based on the stream pull address and edits the first media stream to obtain the second media stream, the method further includes:

[0079] receiving the second media stream sent by the second function; the second media stream represents a media stream used for remote assistance;

[0080] The second media stream is sent to a core network device, so that the core network device provides the second media stream to the first terminal; the core network device includes an IMS.

[0081] Accordingly, the embodiment of the present application further provides another media stream processing method, which is applied to the second function. Figure 2 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 2 ;like Figure 2 As shown, the method includes:

[0082] Step 201: Receive the streaming address sent by the service platform.

[0083] In an embodiment of the present application, the streaming address is generated by the service platform. The second function is set on the PC side of the second terminal, wherein the second function represents a remote assistance function module, and the PC side of the second terminal includes but is not limited to a desktop computer, a laptop computer, etc. The service platform represents a media capability platform. The service platform is a media capability platform, which can be developed based on the C++ language, and the media capability platform is mainly responsible for the functions of receiving, pushing and providing streaming addresses for media streams.

[0084] Here, the second function, namely the remote assistance function module, is set on the second terminal, namely the PC of the assisting party, and can be developed based on Python language, ffmpeg-Python library and pyqt5 as the core module for realizing the remote assistance service function, and the remote assistance function module is mainly responsible for the editing function of the media stream.

[0085] In actual application, the business platform carries the stream pull address in the HTTP request and sends the stream pull address to the second function through the HTTP request.

[0086] Based on this, in one embodiment, the receiving stream pull address sent by the service platform includes:

[0087] receiving a second request sent by the service platform; the second request carries the stream pulling address;

[0088] Parsing the second request to obtain the streaming address;

[0089] The receiving a second request sent by the service platform includes:

[0090] A second request sent by the service platform is received through a first communication connection; the first communication connection represents an HTTP connection.

[0091] Here, since the service platform and the second function use HTTP communication, when the service platform sends the streaming address to the second function, it needs to send the second request to the second function through HTTP communication, that is, the HTTP connection established between the service platform and the second function, and then the second function parses the second request after receiving the second request to obtain the streaming address. The streaming address is carried in the second request, and the second request represents an HTTP request.

[0092] Step 202: Acquire a first media stream based on the stream pull address, and edit the first media stream to obtain a second media stream.

[0093] In an embodiment of the present application, the first media stream is provided by the first terminal, and the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal. A call service connection is established between the first terminal and the second terminal, that is, a native call connection is established between the mobile terminal of the first terminal and the mobile terminal of the second terminal.

[0094] Here, the first terminal can be understood as the calling party, that is, the party being assisted (referred to as the assisted party), and the second terminal can be understood as the called party, that is, the party providing assistance (referred to as the assisting party).

[0095] In actual application, when the service platform and the second function successfully negotiate the stream address, the second function obtains the first media stream sent by the service platform. The first media stream is pushed to the service platform by the core network device, and the first media stream is the media stream provided by the first terminal, i.e., the calling party.

[0096] Based on this, in one embodiment, the acquiring the first media stream based on the stream pulling address includes:

[0097] When the service platform and the second function successfully negotiate the pull stream address, sending a third request to the service platform; the third request is used to request to obtain the first media stream;

[0098] The first media stream sent by the service platform in response to the third request is received.

[0099] Here, after receiving the streaming address sent by the business platform, the second function can negotiate with the business platform regarding the streaming address. If the business platform and the second function successfully negotiate the streaming address, the second function sends a request to the business platform for obtaining the first media stream (corresponding to the aforementioned third request). After receiving the third request, the business platform responds to the third request and sends the first media stream to the second function.

[0100] In actual application, the second function can obtain the first media stream by connecting via a real-time streaming protocol (RTSP, Real Time Streaming Protocol).

[0101] Based on this, in one embodiment, the acquiring the first media stream based on the stream pulling address includes:

[0102] Sending a fourth request to the service platform corresponding to the stream pull address; the fourth request is used to request to establish a second communication connection; the second communication connection represents a real-time streaming protocol connection;

[0103] The first media stream is acquired through the second communication connection.

[0104] In actual application, in one embodiment, the editing of the first media stream to obtain the second media stream includes one or more of the following:

[0105] When the first media stream is in an encoded state, decoding the first media stream to obtain the second media stream;

[0106] Freeze the first media stream to obtain the second media stream;

[0107] Perform a marking operation on the first media stream to obtain the second media stream.

[0108] Here, when the first media stream is in an encoded state, the first media stream in an encoded state can be decoded through the ffmpeg-Python library to obtain a second media stream. For example, if the second media stream is a video screen, the image format of the decoded video screen is converted. For example, the image format of the video screen is converted from opecv to pixmap, and the video screen in the converted image format is played in real time. For the freeze screen operation, the one-key freeze screen function can be implemented by pressing a button to stop updating. Marking the first media stream is actually adding various marks to the corresponding position of the first media stream. For example, manually drawing or adding a serial number to the corresponding position of the first media stream.

[0109] In actual application, after acquiring the first media stream and editing the first media stream to obtain the second media stream, the second function can send the second media stream to the service platform, so as to push the second media stream back to the core network device through the service platform.

[0110] Based on this, in one embodiment, after the first media stream is edited to obtain the second media stream, the method further includes:

[0111] Sending the second media stream to the service platform, so that the service platform sends the second media stream to a core network device, and providing the second media stream to the first terminal through the core network device;

[0112] The core network device includes an IMS, and the second media stream represents a media stream used for remote assistance.

[0113] The embodiment of the present application also provides another media stream processing method, which is an interaction method between the service platform and the second function. Figure 3 The following is a flow chart of the media stream processing method according to an embodiment of the present application. Figure 3 ;like Figure 3 As shown, the method includes:

[0114] Step 301: The service platform receives a first signaling sent by a first function; the first signaling includes a first request.

[0115] In this embodiment of the present application, the first request represents a remote assistance request initiated by the first terminal.

[0116] Step 302: The service platform obtains a first media stream in response to the first request.

[0117] In this embodiment of the present application, the first media stream is provided by the first terminal.

[0118] Step 303: The service platform generates a stream pulling address and sends the stream pulling address to the second function.

[0119] In the embodiment of the present application, the second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0120] Step 304: The second function receives the stream pull address sent by the service platform.

[0121] Step 305: The second function obtains the first media stream based on the stream pull address, and edits the first media stream to obtain the second media stream.

[0122] It should be noted that the specific process of the service platform and the second function completing the media stream processing has been described in detail above and will not be repeated here.

[0123] By adopting the technical solution of the embodiment of the present application, the second function is set on the PC side of the second terminal, so that the first media stream provided by the first terminal is forwarded to the PC side of the second terminal, and then the first media stream is edited by the PC side of the second terminal to obtain the second media stream. In this way, there is no need to install any APP, and only the call service needs to be used to realize the remote assistance service function, which lowers the user's usage threshold, avoids the use of APP to provide remote assistance services, and reduces the use cost of the first terminal, thereby providing remote assistance services to users of the first terminal more conveniently and quickly; at the same time, by setting the stream pulling editing function on the PC side of the second terminal, the performance requirements for the second terminal are reduced.

[0124] The present application is described below in conjunction with application examples.

[0125] In response to the above-mentioned problems existing in the implementation method of remote assistance for video calls in related technologies, the remote assistance service implemented in this application under the new 5G call technology dials the native call in the mobile phone, starts the video call service in the native call, collects audio and video streams (corresponding to the aforementioned first media stream), and forwards the audio and video streams collected by the mobile phone (corresponding to the aforementioned first terminal) to the PC of the other party (corresponding to the aforementioned second terminal). The PC then edits the audio and video stream to obtain the edited audio and video stream (corresponding to the aforementioned second media stream), and then pushes the edited audio and video stream to the user's mobile phone (corresponding to the aforementioned first terminal). This can avoid the use of APP to provide remote assistance services, thereby providing users with remote assistance services more conveniently and quickly.

[0126] The remote frozen screen video assistance solution based on 5G new calls proposed in this application mainly involves three modules: SIP signaling server (corresponding to the first function mentioned above), media capability platform (corresponding to the business platform mentioned above), and remote assistance function module (i.e., stream editing function module, corresponding to the second function mentioned above). This video assistance solution mainly dials the video call through the native call on the mobile phone, and after SIP signaling negotiation through the SIP signaling server, pushes the audio and video stream to the media capability platform of the specified address. After receiving the audio and video stream, the media capability platform generates a stream pulling address to provide it to the remote assistance function module for stream pulling, and performs stream editing, pushes the edited audio and video stream to the media capability platform, and then forwards it to the receiving end (corresponding to the first terminal mentioned above). The functions of the above modules are explained below.

[0127] SIP signaling server: developed based on C++ language, responsible for the interaction of SIP signaling and HTTP requests. Since the media capability platform and remote assistance function module use HTTP communication, and IMS (corresponding to the aforementioned core network equipment) uses SIP signaling communication, the SIP signaling server needs to be used as a signaling transfer station. In addition to receiving, parsing, and replying to SIP signaling as specified in the basic CM-IMS session initiation protocol, it also includes sending HTTP requests to the media capability platform and remote assistance function module after parsing the received SIP signaling.

[0128] Media capability platform: Developed based on C++ language, it is responsible for receiving, pushing and providing streaming addresses for audio and video streams, sending the audio and video streams pushed by IMS on demand, generating RTSP streaming addresses, and forwarding the received audio and video stream data packets to the remote assistance function module after negotiation through the RTSP protocol, so that the remote assistance function module can obtain the audio and video streams of the mobile phone (corresponding to the first terminal mentioned above).

[0129] Remote assistance function module: installed on the PC side of the assisting party (corresponding to the aforementioned second terminal), developed based on Python language, ffmpeg-Python library and pyqt5, as the core module for realizing remote assistance service function, responsible for providing media stream editing functions, such as RTSP streaming playback function, freezing screen function and marking function. Among them, the RTSP streaming playback function can receive audio and video streams after RTSP negotiation with the streaming address generated in the media capability platform, decode the audio and video streams into video images through the ffmpeg-Python library, and convert the decoded opecv image format into pixmap, and display the pixmap in real time on the application through the QLabel in pyqt5 to realize the real-time playback function; the one-key freezing screen function is realized by pressing a button to stop updating the pixmap image format; at the same time, the marking function is realized through the QPainter module in pyqt5, such as manual drawing and adding serial numbers.

[0130] It can be seen that this application uses the 5G new call technology framework and relies on the native call on the mobile phone to realize the remote assistance service function. After the video call is triggered by the native call to establish a connection, the audio and video streams provided by the caller and the called party can be pushed on demand to the host-side assistance software of the called party, that is, the PC side, and the audio and video streams provided by the caller can be frozen, marked, and other operations can be performed on the PC side to complete the remote video assistance service function for the caller.

[0131] Figure 4 Schematic diagram of the first terminal and the remote assistance function module of the embodiment of the present application, such as Figure 4 As shown, the left figure is a schematic diagram of the first terminal. In this application, the first terminal is a user-end mobile phone. The user-end mobile phone has a network cable interface, through which it can communicate with other devices, and the user-end mobile phone can check the status of the router. The right figure is a schematic diagram of the remote assistance function module. The remote assistance function module is set on the expert-end host, that is, the remote assistance function module can be set on the PC side of the assisting party. The expert-end host has a network cable interface, through which it can communicate with other devices, and the expert-end host can check the status of the router, wherein the remote assistance function module can obtain the pull stream address and push back the edited media stream based on the push stream address. It should be noted that the editing functions of the media stream, for example, can be a play function, a freeze screen function, a marking function (for example, manually drawing with a brush, adding serial numbers, adding text, etc.), etc.

[0132] Figure 5 FIG. 1 is a flow chart of the remote assistance service process of an embodiment of the present application. Figure 5 As shown, the specific steps of the remote assistance service process of this application are as follows:

[0133] 1. The mobile user (corresponding to the first terminal mentioned above) makes a video call, triggering the IMS to send a SIP signaling to establish a connection

[0134] In a native call, when a mobile user makes a video call, IMS sends an INVITE request to the SIP signaling server. After the SIP signaling server parses the INVITE request to obtain the SIP signaling, it replies to the IMS with 100Trying, 180Ringing, etc., and informs the media capability platform to prepare to receive the media stream pushed by IMS (corresponding to the first media stream mentioned above). Finally, the SIP signaling server replies with 200OK to the IMS, completing the SIP signaling connection process.

[0135] 2. The media capability platform generates a streaming address

[0136] During the remote assistance service, the caller only needs to obtain the media stream for which he is assisted, and does not need to obtain the media stream provided by the called party. Therefore, the media capability platform will discard the media stream after receiving the media stream provided by the called party, and at the same time receive the media stream provided by the caller and generate a stream pulling address, and send the stream pulling address and the edited media stream receiving address to the remote assistance function module through an HTTP request.

[0137] 3. Remote assistance function module provides assistance services and pushes the stream

[0138] The assisting party (corresponding to the aforementioned second terminal) pre-installs the remote assistance function module software on the PC. After receiving the streaming address, the remote assistance function module immediately sends an RTSP connection request to the streaming address, establishes an RTSP connection, and receives the media stream. After receiving the media stream, the decoding and playback are completed on the PC side, and the media stream editing function is completed. The edited image frames are strung into a stream and sent to the designated streaming address, and the media capability platform completes the retweet.

[0139] Figure 6 Schematic diagram of the media stream push process of the embodiment of the present application. Figure 6As shown, taking the media stream as a video stream as an example for exemplary explanation, the caller provides a video stream and uploads the video stream to the IMS, which is transferred through the SIP signaling server. The IMS pushes the video stream provided by the caller to the media capability platform. It should be noted that with the establishment of the video call, the video stream provided by the called party is pushed from the IMS to the media capability platform together with the video stream provided by the caller. However, the media capability platform will discard the video stream provided by the called party and only push the video stream provided by the caller. After receiving the video stream, the media capability platform generates a streaming address. When the media capability platform and the remote assistance function module successfully negotiate the streaming address, the media capability platform pushes the video stream to the remote assistance function module, which edits the video stream and completely forwards the edited video stream through the media capability platform, and finally pushes it to the caller.

[0140] Compared with the methods of the related art, the solution of the present application has the following beneficial effects:

[0141] (1) In response to the remote assistance problem, this application adopts remote assistance based on the 5G new call scenario, that is, it proposes a remote assistance service for video calls based on native calls using 5G new calls, which lowers the user's usage threshold.

[0142] (2) This application does not require the installation of any APP. The video call remote assistance function can be realized by using native calls. The installation of the remote assistance function module and other work are all placed on the assisting party side, which greatly reduces the use cost of the assisted party and brings convenience to users. At the same time, by setting the streaming editing function on the PC side of the assisting party, the problem of difficulty in user interaction in the new 5G call is solved.

[0143] In order to implement the media stream processing method on the service platform side of the embodiment of the present application, the embodiment of the present application also provides a media stream processing device, which is arranged on the service platform. Figure 7 The structure of the media stream processing device of the present invention is shown in FIG. Figure 1 ,like Figure 7 As shown, the device comprises:

[0144] A first receiving unit 71 is configured to receive a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by a first terminal;

[0145] A first acquiring unit 72 is configured to acquire a first media stream in response to the first request; the first media stream is provided by the first terminal;

[0146] A generating unit 73, used to generate a stream pull address;

[0147] The first sending unit 74 is used to send the stream pulling address to the second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain the second media stream; the second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0148] In one embodiment, the first receiving unit 71 is specifically configured to:

[0149] During a video call between the first terminal and the second terminal, receiving a first signaling sent by a first function;

[0150] The video call is triggered by a native call between the first terminal and the second terminal.

[0151] In one embodiment, the first acquisition unit 72 includes a determination subunit and an acquisition subunit; wherein,

[0152] The determining subunit is configured to determine, in response to the first request, whether first information sent by the first function is received; the first information represents a notification message for notifying the service platform to receive the first media stream;

[0153] The acquisition subunit is configured to acquire the first media stream when it is determined that the first information sent by the first function is received.

[0154] In one embodiment, the determining subunit is specifically used to:

[0155] In a case where the first function and the service platform successfully negotiate the first signaling, it is determined that the first information sent by the first function is received.

[0156] In one embodiment, the first sending unit 74 is specifically configured to:

[0157] Send a second request to the second function; the second request carries the stream pulling address.

[0158] In one embodiment, the first sending unit 74 is further specifically configured to:

[0159] A second request is sent to the second function via a first communication connection; the first communication connection represents an HTTP connection.

[0160] In one embodiment, the device further includes: a third receiving unit and a second sending unit; wherein,

[0161] The third receiving unit is configured to send the pull stream address to the second function after the first sending unit 74 sends the pull stream address, so that the second function obtains the first media stream based on the pull stream address, and edits the first media stream to obtain the second media stream, and then receives the second media stream sent by the second function; the second media stream represents a media stream for remote assistance;

[0162] The second sending unit is used to send the second media stream to a core network device, so that the core network device provides the second media stream to the first terminal; the core network device includes an IMS.

[0163] In one embodiment, the device further includes: a third acquisition unit and a discarding unit; wherein,

[0164] The third acquisition unit is configured to acquire a third media stream when the first acquisition unit 72 acquires the first media stream;

[0165] The discarding unit is configured to discard the third media stream when it is detected that the third media stream is provided by the second terminal.

[0166] In actual application, the first receiving unit 71, the first acquiring unit 72 and the first sending unit 74 can be implemented by a communication interface in the media stream processing device, and the generating unit 73 can be implemented by a processor in the media stream processing device.

[0167] It should be noted that: the media stream processing device provided in the above embodiment only uses the division of the above program modules as an example when performing media stream processing. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the media stream processing device provided in the above embodiment and the media stream processing method embodiment on the service platform side belong to the same concept. The specific implementation process is detailed in the media stream processing method embodiment on the service platform side, which will not be repeated here.

[0168] In order to implement the media stream processing method on the second function side of the embodiment of the present application, the embodiment of the present application further provides another media stream processing device, which is arranged on the second function. Figure 8 The structure of the media stream processing device of the present invention is shown in FIG. Figure 2 ,like Figure 8 As shown, the device comprises:

[0169] The second receiving unit 81 is used to receive a stream pull address sent by the service platform; the stream pull address is generated by the service platform;

[0170] A second acquisition unit 82, configured to acquire a first media stream based on the stream pull address;

[0171] The editing unit 83 is configured to edit the first media stream to obtain a second media stream; the first media stream is provided by the first terminal, and the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0172] The second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0173] In one embodiment, the second receiving unit 81 includes a receiving subunit and a parsing subunit; wherein,

[0174] The receiving subunit is configured to receive a second request sent by the service platform; the second request carries the stream pulling address;

[0175] The parsing subunit is used to parse the second request to obtain the stream pulling address;

[0176] Furthermore, the receiving subunit is specifically used for:

[0177] A second request sent by the service platform is received through a first communication connection; the first communication connection represents an HTTP connection.

[0178] In one embodiment, the second obtaining unit 82 is specifically configured to:

[0179] When the service platform and the second function successfully negotiate the pull stream address, sending a third request to the service platform; the third request is used to request to obtain the first media stream;

[0180] The first media stream sent by the service platform in response to the third request is received.

[0181] In one embodiment, the second obtaining unit 82 is further specifically configured to:

[0182] Sending a fourth request to the service platform corresponding to the stream pull address; the fourth request is used to request to establish a second communication connection; the second communication connection represents a real-time streaming protocol connection;

[0183] The first media stream is acquired through the second communication connection.

[0184] In one embodiment, the editing unit 83 is specifically used for one or more of the following:

[0185] When the first media stream is in an encoded state, decoding the first media stream to obtain the second media stream;

[0186] Freeze the first media stream to obtain the second media stream;

[0187] Perform a marking operation on the first media stream to obtain the second media stream.

[0188] In one embodiment, the device further includes a third sending unit; wherein,

[0189] The third sending unit is configured to, after the editing unit 83 edits the first media stream to obtain a second media stream, send the second media stream to the service platform, so that the service platform sends the second media stream to the core network device, and provides the second media stream to the first terminal through the core network device;

[0190] The core network device includes an IMS, and the second media stream represents a media stream used for remote assistance.

[0191] In actual application, the second receiving unit 81 and the second obtaining unit 82 may be implemented by a communication interface in the media stream processing device, and the editing unit 83 may be implemented by a processor in the media stream processing device.

[0192] It should be noted that: the media stream processing device provided in the above embodiment only uses the division of the above program modules as an example when performing media stream processing. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the media stream processing device provided in the above embodiment and the media stream processing method embodiment of the second functional side belong to the same concept. The specific implementation process is detailed in the media stream processing method embodiment of the second functional side, which will not be repeated here.

[0193] Based on the hardware implementation of the above program modules, and in order to implement the media stream processing method on the service platform side of the embodiment of the present application, the embodiment of the present application also provides a service platform, Fig. 9 The hardware structure diagram of the service platform of the embodiment of the present application is as follows: Fig. 9 As shown, the service platform 900 includes:

[0194] The first communication interface 901 is capable of information exchange with the second function;

[0195] The first processor 902 is connected to the first communication interface 901 to implement information interaction with the second function, and is used to execute the method provided by one or more technical solutions on the above-mentioned business platform side when running the computer program, and the computer program is stored in the first memory 903.

[0196] Specifically, the first communication interface 901 is used to receive a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by a first terminal; in response to the first request, a first media stream is obtained; the first media stream is provided by the first terminal;

[0197] The first processor 902 is used to generate a stream pull address;

[0198] The first communication interface 901 is also used to send the stream pulling address to the second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain the second media stream; the second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0199] In one embodiment, the first communication interface 901 is specifically used for:

[0200] During a video call between the first terminal and the second terminal, receiving a first signaling sent by a first function;

[0201] The video call is triggered by a native call between the first terminal and the second terminal.

[0202] In one embodiment, the first communication interface 901 is specifically used for:

[0203] In response to the first request, determining whether first information sent by the first function is received; the first information represents a notification message for notifying the service platform to receive the first media stream;

[0204] When it is determined that the first information sent by the first function is received, the first media stream is acquired.

[0205] In one embodiment, the first communication interface 901 is specifically used for:

[0206] In a case where the first function and the service platform successfully negotiate the first signaling, it is determined that the first information sent by the first function is received.

[0207] In one embodiment, the first communication interface 901 is specifically used for:

[0208] Sending a second request to the second function; the second request carries the stream pulling address;

[0209] The first communication interface 901 is further specifically used for:

[0210] A second request is sent to the second function via a first communication connection; the first communication connection represents an HTTP connection.

[0211] In one embodiment, the first communication interface 901 is further used for receiving the second media stream sent by the second function after the pull stream address is sent to the second function so that the second function obtains the first media stream based on the pull stream address and edits the first media stream to obtain the second media stream; the second media stream represents a media stream for remote assistance;

[0212] The first communication interface 901 is further used to send the second media stream to a core network device, so that the core network device provides the second media stream to the first terminal; the core network device includes an IMS.

[0213] In one embodiment, the first communication interface 901 is further used to obtain a third media stream when obtaining the first media stream;

[0214] The first processor 902 is further configured to discard the third media stream when detecting that the third media stream is provided by the second terminal.

[0215] It should be noted that the specific processing process of the first communication interface 901 and the first processor 902 can be understood by referring to the media stream processing method on the service platform side.

[0216] Of course, in actual application, the various components in the service platform 900 are coupled together through the first bus system 904. It can be understood that the first bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig. 9 In the figure, various buses are labeled as a first bus system 904.

[0217] The first memory 903 in the embodiment of the present application is used to store various types of data to support the operation of the service platform 900. Examples of such data include: any computer program used to operate on the service platform 900.

[0218] The media stream processing method on the service platform side disclosed in the above embodiment of the present application can be applied to the first processor 902, or implemented by the first processor 902. The first processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the media stream processing method on the service platform side can be completed by the hardware integrated logic circuit or software instructions in the first processor 902. The above-mentioned first processor 902 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 902 can implement or execute the media stream processing methods, steps and logic block diagrams on the service platform side disclosed in the embodiment of the present application. The general processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the media stream processing method on the service platform side disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the first memory 903. The first processor 902 reads the information in the first memory 903 and completes the steps of the media stream processing method on the aforementioned service platform side in combination with its hardware.

[0219] In an exemplary embodiment, the service platform 900 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned media stream processing method on the service platform side.

[0220] Based on the hardware implementation of the above program module, and in order to implement the media stream processing method of the second functional side of the embodiment of the present application, the embodiment of the present application also provides a second function, Fig.10 This is a schematic diagram of the hardware structure of the second function of the embodiment of the present application, such as Fig.10 As shown, the second function 1000 includes:

[0221] The second communication interface 1001 is capable of exchanging information with the service platform;

[0222] The second processor 1002 is connected to the second communication interface 1001 to implement information interaction with the business platform, and is used to execute the methods provided by one or more technical solutions of the above-mentioned second functional side when running a computer program, and the computer program is stored in the second memory 1003.

[0223] Specifically, the second communication interface 1001 is used to receive a stream pull address sent by a service platform; the stream pull address is generated by the service platform; and the first media stream is obtained based on the stream pull address;

[0224] The second processor 1002 is configured to edit the first media stream to obtain a second media stream; the first media stream is provided by the first terminal, and the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal;

[0225] The second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0226] In one embodiment, the second communication interface 1001 is specifically used for:

[0227] receiving a second request sent by the service platform; the second request carries the stream pulling address;

[0228] Parsing the second request to obtain the streaming address;

[0229] Furthermore, the second communication interface 1001 is specifically used for:

[0230] A second request sent by the service platform is received through a first communication connection; the first communication connection represents an HTTP connection.

[0231] In one embodiment, the second communication interface 1001 is specifically used for:

[0232] When the service platform and the second function successfully negotiate the pull stream address, sending a third request to the service platform; the third request is used to request to obtain the first media stream;

[0233] The first media stream sent by the service platform in response to the third request is received.

[0234] In one embodiment, the second communication interface 1001 is specifically used for:

[0235] Sending a fourth request to the service platform corresponding to the stream pull address; the fourth request is used to request to establish a second communication connection; the second communication connection represents a real-time streaming protocol connection;

[0236] The first media stream is acquired through the second communication connection.

[0237] In one embodiment, the second processor 1002 is specifically configured to:

[0238] When the first media stream is in an encoded state, decoding the first media stream to obtain the second media stream;

[0239] Freeze the first media stream to obtain the second media stream;

[0240] Perform a marking operation on the first media stream to obtain the second media stream.

[0241] In one embodiment, the second communication interface 1001 is further configured to, after the second processor 1002 edits the first media stream to obtain a second media stream, send the second media stream to the service platform, so that the service platform sends the second media stream to a core network device, and provides the second media stream to the first terminal through the core network device;

[0242] The core network device includes an IMS, and the second media stream represents a media stream used for remote assistance.

[0243] It should be noted that the specific processing process of the second communication interface 1001 can be understood by referring to the media stream processing method of the second functional side.

[0244] Of course, in actual application, the various components in the second function 1000 are coupled together through the second bus system 1004. It can be understood that the second bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.10 Various buses are labeled as a second bus system 1004 .

[0245] The second memory 1003 in the embodiment of the present application is used to store various types of data to support the operation of the second function 1000. Examples of such data include: any computer program used to operate on the second function 1000.

[0246] The media stream processing method of the second functional side disclosed in the above embodiment of the present application can be applied to the second processor 1002, or implemented by the second processor 1002. The second processor 1002 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the media stream processing method of the second functional side can be completed by the hardware integrated logic circuit or software instructions in the second processor 1002. The above second processor 1002 can be a general processor, DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The second processor 1002 can implement or execute the media stream processing methods, steps and logic block diagrams of each second functional side disclosed in the embodiment of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the media stream processing method of the second functional side disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium located in the second memory 1003. The second processor 1002 reads the information in the second memory 1003 and completes the steps of the media stream processing method of the second functional side in combination with its hardware.

[0247] In an exemplary embodiment, the second function 1000 can be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to execute the media stream processing method on the aforementioned second function side.

[0248] It can be understood that the memory (including the first memory 903 and the second memory 1003) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0249] In order to implement the media stream processing method of the embodiment of the present application, the embodiment of the present application also provides a media stream processing system. Fig.11 The structure diagram of the media stream processing system of the embodiment of the present application is as follows: Fig.11 As shown, the system includes: a business platform 900 and a second function 1000; wherein,

[0250] The service platform 900 is configured to receive a first signaling sent by a first function, wherein the first signaling includes a first request; obtain a first media stream in response to the first request; generate a stream pull address, and send the stream pull address to the second function 1000;

[0251] The second function 1000 is used to receive a stream pulling address sent by the service platform 900; obtain the first media stream based on the stream pulling address, and edit the first media stream to obtain a second media stream.

[0252] Here, the first request represents a remote assistance request initiated by the first terminal; the first media stream is provided by the first terminal; the second function is set on the PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

[0253] It should be noted that the specific process of the service platform 900 and the second function 1000 completing the media stream processing has been described in detail above and will not be repeated here.

[0254] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 903 storing a computer program, the computer program can be executed by the first processor 902 in the service platform 900 to complete the steps of the media stream processing method on the service platform side, or, including a second memory 1003 storing a computer program, the computer program can be executed by the second processor 1002 in the second function 1000 to complete the steps of the media stream processing method on the second function side. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0255] In an exemplary embodiment, the embodiment of the present application also provides a computer program product, including a computer program, which can be executed by the first processor 902 in the service platform 900 to complete the steps of the media stream processing method on the service platform side, or the computer program can be executed by the second processor 1002 in the second function 1000 to complete the steps of the media stream processing method on the second function side.

[0256] It should be noted that: "first", "second", "third", etc. are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0257] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0258] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A media stream processing method, characterized in that: Applied to a business platform, the method includes: receiving a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by the first terminal; In response to the first request, obtaining a first media stream; the first media stream is provided by the first terminal; Generate a streaming address and send the streaming address to a second function, so that the second function obtains the first media stream based on the streaming address and edits the first media stream to obtain a second media stream; the second function is set on the personal computer PC side of the second terminal, and a call service connection is established between the first terminal and the second terminal.

2. The method according to claim 1, characterized in that: The receiving a first signaling sent by a first function includes: During a video call between the first terminal and the second terminal, receiving a first signaling sent by a first function; The video call is triggered by a native call between the first terminal and the second terminal.

3. The method according to claim 1, characterized in that The step of obtaining a first media stream in response to the first request includes: In response to the first request, determining whether first information sent by the first function is received; the first information represents a notification message for notifying the service platform to receive the first media stream; When it is determined that the first information sent by the first function is received, the first media stream is acquired.

4. The method according to claim 3, characterized in that: The determining whether the first information sent by the first function is received includes: In a case where the first function and the service platform successfully negotiate the first signaling, it is determined that the first information sent by the first function is received.

5. The method according to claim 1, characterized in that The sending the stream pull address to the second function includes: Sending a second request to the second function; the second request carries the stream pulling address; The sending a second request to the second function includes: A second request is sent to the second function via a first communication connection; the first communication connection represents a Hypertext Transfer Protocol HTTP connection.

6. The method according to claim 1, characterized in that After sending the stream pull address to the second function so that the second function obtains the first media stream based on the stream pull address and edits the first media stream to obtain the second media stream, the method further includes: receiving the second media stream sent by the second function; the second media stream represents a media stream used for remote assistance; The second media stream is sent to a core network device, so that the core network device provides the second media stream to the first terminal; the core network device includes an IP multimedia subsystem IMS.

7. The method according to claim 1, characterized in that The method further comprises: When the first media stream is obtained, a third media stream is obtained; In a case where it is detected that the third media stream is provided by the second terminal, the third media stream is discarded.

8. A media stream processing method, characterized in that: Applied to the second function, the method comprises: Receive a stream pull address sent by a service platform; the stream pull address is generated by the service platform; A first media stream is obtained based on the stream pull address, and the first media stream is edited to obtain a second media stream; the first media stream is provided by a first terminal, and the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal; The second function is set at the personal computer PC of the second terminal, and a call service connection is established between the first terminal and the second terminal.

9. The method according to claim 8, characterized in that The receiving service platform sends a streaming address, including: receiving a second request sent by the service platform; the second request carries the stream pulling address; Parsing the second request to obtain the streaming address; The receiving a second request sent by the service platform includes: A second request sent by the service platform is received through a first communication connection; the first communication connection represents a Hypertext Transfer Protocol HTTP connection.

10. The method according to claim 8, characterized in that The acquiring the first media stream based on the stream pulling address includes: When the service platform and the second function successfully negotiate the pull stream address, sending a third request to the service platform; the third request is used to request to obtain the first media stream; The first media stream sent by the service platform in response to the third request is received.

11. The method according to claim 10, characterized in that The acquiring the first media stream based on the stream pulling address includes: Sending a fourth request to the service platform corresponding to the stream pull address; the fourth request is used to request to establish a second communication connection; the second communication connection represents a real-time streaming protocol connection; The first media stream is acquired through the second communication connection.

12. The method according to claim 8, characterized in that The editing of the first media stream to obtain the second media stream includes one or more of the following: When the first media stream is in an encoded state, decoding the first media stream to obtain the second media stream; Freeze the first media stream to obtain the second media stream; Perform a marking operation on the first media stream to obtain the second media stream.

13. The method according to claim 8, characterized in that After the first media stream is edited to obtain a second media stream, the method further includes: Sending the second media stream to the service platform, so that the service platform sends the second media stream to a core network device, and providing the second media stream to the first terminal through the core network device; The core network device includes an IP Multimedia Subsystem IMS, and the second media stream represents a media stream used for remote assistance.

14. A media stream processing device, characterized in that: Applied to a business platform, the device comprises: A first receiving unit, configured to receive a first signaling sent by a first function; the first signaling includes a first request, and the first request represents a remote assistance request initiated by the first terminal; A first acquiring unit, configured to acquire a first media stream in response to the first request; the first media stream is provided by the first terminal; A generating unit, used to generate a stream pulling address; The first sending unit is used to send the stream pulling address to the second function, so that the second function obtains the first media stream based on the stream pulling address, and edits the first media stream to obtain the second media stream; the second function is set on the personal computer PC end of the second terminal, and a call service connection is established between the first terminal and the second terminal.

15. A media stream processing device, characterized in that: Applied to the second function, the device comprises: A second receiving unit is used to receive a stream pulling address sent by a service platform; the stream pulling address is generated by the service platform; A second acquisition unit, configured to acquire a first media stream based on the stream pull address; an editing unit, configured to edit the first media stream to obtain a second media stream; the first media stream is provided by a first terminal, the first media stream is obtained by the service platform in response to a first request, and the first request represents a remote assistance request initiated by the first terminal; The second function is set at the personal computer PC of the second terminal, and a call service connection is established between the first terminal and the second terminal.

16. A business platform, characterized in that: include: a first processor and a first memory for storing a computer program executable on said first processor; Wherein, when the first processor is used to run the computer program, the steps of the method described in any one of claims 1 to 7 are executed.

17. A second function, characterized in that, include: a second processor and a second memory for storing a computer program executable on the second processor; Wherein, when the second processor is used to run the computer program, the steps of the method described in any one of claims 8 to 13 are executed.

18. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7, or implements the steps of the method according to any one of claims 8 to 13.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7, or implements the steps of the method according to any one of claims 8 to 13.

Images