Screen video sharing method and device, electronic equipment and storage medium

Abstract

The invention provides a screen video sharing method and device, electronic equipment and a storage medium, and belongs to the technical field of communication, and the method comprises the steps that a call is established with a video customer service through an SIP communication protocol, the SIP communication protocol comprises identification field information, and the identification field information represents whether terminal equipment supports remote assistance or not; if the identification field information represents that the terminal equipment supports remote assistance, after a remote assistance initiating instruction input by a user is received, a video source of the call is set as a screenshot video stream of the terminal equipment, and the remote assistance initiating instruction is an instruction for instructing the terminal equipment to initiate remote assistance. According to the invention, the screen of the terminal equipment is shared to the video customer service through remote assistance, so that the customer service can quickly position the problem; according to the method, the identification field is carried in the SIP protocol, whether remote assistance is supported or not is actively notified, functional abnormity caused by the fact that part of terminals do not support remote assistance is avoided, and the problem that page operation is complex due to multiple times of signaling interaction is solved.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular to a screen video sharing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of mobile communications and the Internet industry, various mobile video applications have become popular, and users have gradually developed the habit of using videos. When users encounter problems, they no longer need to seek help from relatives offline or find special staff. They can directly call video customer service for help. Video customer service has become an important customer service channel.

[0003] Currently, video customer service can provide voice calls and video calls, but for problems with the phone itself, such as a certain function not being able to operate, an APP not being able to be used, an error on a certain page, etc., the problem can only be located through the user's verbal description, which is not conducive to the handling and resolution of the problem.

[0004] Therefore, there is an urgent need for a remote assistance method to share the user's mobile phone screen with customer service personnel in real time to quickly locate and solve the problem. Summary of the invention

[0005] The present invention provides a screen video sharing method, device, electronic device and storage medium, which are used to solve the defect in the prior art that video customer service can only locate problems through user's language description, which is not conducive to problem handling.

[0006] The present invention provides a screen video sharing method, comprising the following steps: Initiating a call request to the video customer service through the SIP communication protocol, or accepting a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; If the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device to share the screenshot of the terminal device to the video customer service in real time. The remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0007] According to a screen video sharing method provided by the present invention, after initiating a call request to a video customer service through the SIP communication protocol, or accepting a call request initiated by the video customer service through the SIP communication protocol, the method further includes: During an audio call or a one-way video call, graying out the remote assistance component or hiding the remote assistance component; During a two-way video call, highlighting the remote assistance component; The remote assistance initiation instruction is an operation instruction for the remote assistance component.

[0008] According to a screen video sharing method provided by the present invention, after receiving a remote assistance initiation instruction input by a user, setting the video source of the call to the screenshot video stream of the terminal device includes: After receiving the remote assistance initiation instruction input by the user, a remote assistance confirmation prompt box is displayed; After receiving the remote assistance confirmation initiation instruction input by the user through the remote assistance confirmation prompt box, a remote assistance connection is established and the video source of the call is set to the screenshot video stream of the terminal device. The remote assistance confirmation initiation instruction is an instruction to confirm the initiation of remote assistance.

[0009] According to a screen video sharing method provided by the present invention, after establishing a remote auxiliary connection and setting the video source of the call to the screenshot video stream of the terminal device, the method further includes: hiding a floating window for receiving a video stream, wherein the received video stream is a video stream received from the video customer service; The call timing duration and remote assistance timing duration are displayed in a carousel based on the preset time interval.

[0010] According to a screen video sharing method provided by the present invention, after establishing a remote auxiliary connection and setting the video source of the call to the screenshot video stream of the terminal device, the method further includes: After receiving an application switching instruction input by a user, displaying a floating window for receiving the video stream, wherein the application switching instruction is an instruction for instructing the terminal device to switch from a call page to another application page; In response to the user's dragging operation, the floating window for receiving the video stream is collapsed, and a floating window folding icon is displayed at the edge of the screen; In response to a click operation on the floating window folding icon, the floating window folding icon is collapsed, and the floating window for receiving the video stream is displayed.

[0011] According to a screen video sharing method provided by the present invention, after receiving a remote assistance initiation instruction input by a user, the method further includes: Based on the screen parameters of the terminal device, generate fmtp parameters, wherein the fmtp parameters include a screen proportional scaling resolution of the terminal device; Sending a session initiation request to the video customer service, wherein the session initiation request includes the fmtp parameter; After receiving the response information from the video customer service, the original screen resolution video is scaled proportionally, and the scaled video stream is sent to the video customer service, the response information indicates that the session initiation request has been successfully processed, and the response information includes a resolution that is the same as the screen proportional scaling resolution.

[0012] The present invention also provides a screen video sharing device, comprising the following modules: A call establishment module, used to: initiate a call request to the video customer service through the SIP communication protocol, or accept a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; A video source setting module is used to: if the identification field information indicates that the terminal device supports remote assistance, then after receiving a remote assistance initiation instruction input by the user, set the video source of the call to the screenshot video stream of the terminal device, so as to share the screenshot of the terminal device to the video customer service in real time, and the remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned screen video sharing methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the screen video sharing method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the screen video sharing method described in any one of the above is implemented.

[0016] The screen video sharing method, device, electronic device and storage medium provided by the present invention initiate a call request to the video customer service through the SIP communication protocol, or accept a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, wherein the identification field information indicates whether the terminal device supports remote assistance; if the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device, wherein the remote assistance initiation instruction is an instruction to instruct the terminal device to initiate remote assistance. The present invention shares the screen of the terminal device with the video customer service through remote assistance, which is conducive to the video customer service to quickly locate and solve the problem; in addition, the identification field information is carried in the SIP communication protocol, so that the terminal device actively notifies whether its native call application supports the remote assistance function, which, on the one hand, avoids the problem of functional abnormality of some terminals due to non-support of remote assistance, and on the other hand, does not require additional complex interaction logic before screen video sharing, thereby solving the problem of complex interaction logic and complex page operation caused by multiple signaling interactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the screen video sharing method provided by the present invention; Figure 2 This is the second flow chart of the screen video sharing method provided by the present invention; Figure 3 It is a schematic diagram of a process of a user terminal device calling a video customer service provided by the present invention; Figure 4 It is a schematic diagram of the process of video customer service outbound calling a user terminal device provided by the present invention; Figure 5 It is a schematic diagram of the terminal device architecture provided by the present invention; Figure 6 is a schematic diagram of confirming a remote assistance prompt provided by the present invention; Figure 7 is a schematic diagram of establishing a remote auxiliary connection provided by the present invention; Figure 8 It is a schematic diagram of the completion of the remote auxiliary connection provided by the present invention; Fig. 9This is a schematic diagram of the remote auxiliary timing duration provided by the present invention; Fig.10 is a schematic diagram of switching applications during remote assistance provided by the present invention; Fig.11 It is a schematic diagram of folding the video floating window provided by the present invention; Fig.12 is a schematic diagram of the end of the remote assistance provided by the present invention; Fig.13 It is a schematic diagram of the process of video stream switching provided by the present invention; Fig.14 It is a structural schematic diagram of the screen video sharing device provided by the present invention; Fig.15 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be a connection between two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0021] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" means at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0022] Combine the following Figure 1-Figure 15 The screen video sharing method, device, electronic device and storage medium provided by the embodiments of the present invention are described.

[0023] Figure 1 It is one of the flow charts of the screen video sharing method provided by the present invention, such as Figure 1 As shown, the method includes the following: S110, initiating a call request to the video customer service through the SIP communication protocol, or accepting the call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; S120, if the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device, so as to share the screenshot of the terminal device to the video customer service in real time, and the remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0024] It should be noted that the screen video sharing method provided in the embodiments of the present invention is executed by a user terminal device, specifically, a mobile phone, a tablet computer, a television, a laptop computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable device, a virtual reality device, etc. For ease of understanding, the following embodiments take the mobile phone as an example to illustrate the screen video sharing method provided by the present invention.

[0025] The establishment of a call depends on the SIP communication protocol. The user terminal needs to initiate a call request to the video customer service through the SIP protocol. After the video customer service agrees, the two parties can talk. This is the basis for the call between the two devices.

[0026] In S110, the SIP communication protocol used when establishing a call is reused. By carrying identification field information in the SIP protocol, the terminal device is informed whether its native call application supports remote assistance, screen sharing or other functions. This not only avoids functional abnormalities in some terminals due to non-support of remote assistance, but also solves the problems of complex interaction logic and page operations caused by multiple signaling interactions.

[0027] It is understandable that the identification field information carried in the SIP protocol can also be extended to other functions of the native call application of the terminal device, such as multi-party conferencing, graffiti, real-time translation, touch screen interaction, etc.

[0028] In the embodiment of the present invention, when the video customer service exchanges signals with the user terminal to establish a session, the video customer service adds a remote assistance tag line (a=content:g.3gpp.cmos) in the SDP; the user terminal needs to identify the tag line and resend it to the video customer service after processing, thus completing the notification of whether the devices of both parties support the remote assistance function. Specifically, the establishment of a call between the terminal device and the video customer service can be divided into two categories, one is the user terminal device calling the video customer service, and the other is the video customer service calling the user terminal device: Figure 3 FIG. 1 is a flow chart of a user terminal device calling a video customer service provided by the present invention. Figure 3 As shown in the figure, for a call initiated by a user terminal device, after receiving the 200OK of the video customer service and sending ACK, the terminal layer passes the content of the tag line (content:g.3gpp.cmos) to the chip layer; the chip layer parses the content of the tag line; when the chip layer finds that the content contains "content:g.3gpp.cmos", the chip layer sends INFO (the content contains "ack:g.3gpp.cmos"), which is encapsulated by the terminal layer (Content-Type:application / text) and sent to the video customer service; Figure 4 FIG. 1 is a flow chart of a video customer service outbound call to a user terminal device provided by the present invention, such as Figure 4 As shown in the figure, for a call made by the video customer service to the user's terminal device, the terminal layer sends a 200OK. After receiving the ACK from the video customer service, the terminal layer passes the content of the tag line (content:g.3gpp.cmos) to the chip layer; the chip layer parses the content of the tag line; when the chip layer finds that the content contains "content:g.3gpp.cmos", the chip layer triggers the sending of INFO (the content contains "ack:g.3gpp.cmos"), which is encapsulated by the terminal layer (Content-Type:application / text) and sent to the video customer service.

[0029] It is understandable that during the signaling negotiation process, the terminal layer of the terminal device needs to be able to parse the a=content:g.3gpp.cmos line carried in the SDP sent by the video customer service. When the session is established, the terminal layer adds the line information (content:g.3gpp.cmos) carried by the above SDP to the session establishment event pushed to the chip layer. The terminal layer can provide an interface, and the chip layer can call this interface to set and send an INFO message (Content-Type is application / text).

[0030] It is understandable that for the chip of the terminal device, the chip can identify the line information (content:g.3gpp.cmos) carried in the SDP as the video customer service supports the remote assistance function through the session establishment event uploaded by the terminal, and determine that if the terminal also supports the remote assistance function, it can reply with the INFO content of "ack:g.3gpp.cmos".

[0031] Optionally, the format of the INFO message sent by the user terminal to the video customer service remote assistance function response is as follows: Content-Type:application / text

[0032] <?xml version="1.0" encoding="UTF-8" ?>

[0033] <msml version="1.1">

[0034] <send event="ack:g.3gpp.cmos" / >

[0035] < / msml>

[0036] In S120, the terminal device implements the Yuncheng auxiliary function and needs to rely on the chip to provide the ability to switch video sources.

[0037] Alternatively, you can use the AOSP (Android Open Source Project) API to implement this. Use the setCamera(String cameraId) API provided by the MTK chip to pass the cameraId to the fixed string "screen" to switch the video source to the screen capture video stream. To end screen sharing, you can reset the camera to the valid camera Id.

[0038] In the specific implementation process, the screen sharing function is controlled by carrierconfig (a component in the Android operating system used to manage operator-specific settings and configurations) "mtk_key_vt_screen_share_support_bool". If you need to turn on the screen sharing function, you need to set the carrierconfig to true. During the screen sharing process, since the screen is always shared in portrait mode, the terminal UI needs to block the method of setting the device orientation during the screen sharing process. In Android development, setDeviceOrientation() is blocked. Even if the UI detects that the user has turned the screen, do not call this method (only during the screen sharing process).

[0039] Figure 5 is a schematic diagram of the terminal device architecture provided by the present invention, such as Figure 5 As shown in the figure, the chip manufacturer's Modem / Media chip layer is responsible for adding remote assistance capabilities, and the Android system layer is responsible for expanding the AOSP interface capabilities. The terminal manufacturer is responsible for the development and design of native call applications, with a focus on adding remote assistance components to the native call UI interface. The terminal and the chip interact through the IMSSIP interface.

[0040] The screen video sharing method provided by the embodiment of the present invention initiates a call request to the video customer service through the SIP communication protocol, or accepts the call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, wherein the identification field information indicates whether the terminal device supports remote assistance; if the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device, wherein the remote assistance initiation instruction is an instruction to instruct the terminal device to initiate remote assistance. The present invention shares the screen of the terminal device with the video customer service through remote assistance, which is conducive to the video customer service to quickly locate and solve the problem; in addition, the identification field information is carried in the SIP communication protocol, so that the terminal device actively notifies whether its native call application supports the remote assistance function, which, on the one hand, avoids the problem of functional abnormality of some terminals due to non-support of remote assistance, and on the other hand, does not require additional complex interaction logic before screen video sharing, thereby solving the problem of complex interaction logic and complex page operation caused by multiple signaling interactions.

[0041] In an optional embodiment, after initiating a call request to the video customer service through the SIP communication protocol, or accepting a call request initiated by the video customer service through the SIP communication protocol, the method further includes: During an audio call or a one-way video call, graying out the remote assistance component or hiding the remote assistance component; During a two-way video call, highlighting the remote assistance component; The remote assistance initiation instruction is an operation instruction for the remote assistance component.

[0042] Specifically, remote assistance relies on two-way video calls. During audio calls or one-way video calls, the remote assistance component must be hidden or grayed out, and users cannot operate the remote assistance component. Only in the two-way video call state will the remote assistance component be highlighted, and users can operate the remote assistance component to initiate remote assistance.

[0043] Here, the remote assistance initiation instruction can be a user's click operation on the remote assistance component, or a long press of the remote assistance component, etc. The specific operation method is not limited here.

[0044] In an optional embodiment, after receiving the remote assistance initiation instruction input by the user, setting the video source of the call to the screenshot video stream of the terminal device includes: After receiving the remote assistance initiation instruction input by the user, a remote assistance confirmation prompt box is displayed; After receiving the remote assistance confirmation initiation instruction input by the user through the remote assistance confirmation prompt box, a remote assistance connection is established and the video source of the call is set to the screenshot video stream of the terminal device. The remote assistance confirmation initiation instruction is an instruction to confirm the initiation of remote assistance.

[0045] Figure 6 It is a schematic diagram of confirming the remote assistance prompt provided by the present invention; Figure 7 is a schematic diagram of establishing a remote auxiliary connection provided by the present invention; like Figure 6-Figure 7 As shown in the figure, in a two-way video call, users can initiate remote assistance through the remote assistance component on the native call application of the terminal device. Since remote assistance will share the screen with the other party, there is a security risk. After the user initiates remote assistance, a remote assistance confirmation prompt box will pop up. The remote assistance confirmation prompt box includes a component indicating confirmation to initiate remote assistance and a component to cancel remote assistance to avoid user misoperation. After the user confirms, the connection is established and the connecting icon is displayed. After the connection is successful, the video customer service can share the screen of the user's terminal device.

[0046] In the embodiment of the present invention, the remote assistance confirmation initiation instruction may be a click operation or a long press operation on the component indicating confirmation to initiate remote assistance in the remote assistance confirmation prompt box.

[0047] Understandably, Figure 6This is only an exemplary diagram. The specific text content displayed in the remote assistance confirmation prompt box can be determined according to actual usage requirements. It only needs to prompt the user to confirm whether to turn on remote assistance.

[0048] The screen video sharing method provided by the embodiment of the present invention pops up a remote assistance confirmation prompt box after initiating remote assistance through the remote assistance component on the native call application of the terminal device to avoid user misoperation.

[0049] In an optional embodiment, after establishing the remote auxiliary connection and setting the video source of the call as the screenshot video stream of the terminal device, the method further includes: hiding a floating window for receiving a video stream, wherein the received video stream is a video stream received from the video customer service; The call timing duration and remote assistance timing duration are displayed in a carousel based on the preset time interval.

[0050] Figure 8 Schematic diagram of the remote auxiliary connection provided by the present invention. Figure 8 As shown, after remote assistance is established, in order to ensure the integrity of the shared content, the floating window in the upper right corner of the camera in the original two-way video is hidden.

[0051] Fig. 9 Schematic diagram of the remote auxiliary timing duration provided by the present invention. Fig. 9 As shown, after remote assistance is established, in order to improve the remote assistance interactive experience, a remote assistance timer is added to the call timer, which is displayed in a carousel with the call duration (the carousel time interval can be set by the terminal device itself).

[0052] Optionally, the call timing duration and the remote assistance timing duration are displayed in an alternating scrolling manner.

[0053] like Figure 7 and Figure 8 As shown, the fourth component from left to right is the remote assistance component. Before and after the remote assistance connection is established, the icon of the remote assistance component is displayed differently to facilitate distinguishing whether the remote assistance is turned on.

[0054] The screen video sharing method provided by the embodiment of the present invention hides the video floating window after establishing remote assistance to ensure the integrity of the shared content; and displays the call timing duration and the remote assistance timing duration in a carousel to enhance the remote assistance interactive experience.

[0055] In an optional embodiment, after establishing the remote auxiliary connection and setting the video source of the call as the screenshot video stream of the terminal device, the method further includes: After receiving an application switching instruction input by a user, displaying a floating window for receiving the video stream, wherein the application switching instruction is an instruction for instructing the terminal device to switch from a call page to another application page; In response to the user's dragging operation, the floating window for receiving the video stream is collapsed, and a floating window folding icon is displayed at the edge of the screen; In response to a click operation on the floating window folding icon, the floating window folding icon is collapsed, and the floating window for receiving the video stream is displayed.

[0056] Fig.10 is a schematic diagram of switching applications during remote assistance provided by the present invention; Fig.11 It is a schematic diagram of folding the video floating window provided by the present invention; like Fig.10 and Fig.11 As shown, when the user switches from the call page to other applications, the camera thumbnail floating window in the two-way video needs to be redisplayed and the screen content is updated in real time; when the user drags the floating window to the edge of the screen, the floating window is folded and an arrow-shaped floating window folding icon is displayed; click the floating window folding icon again to redisplay the video floating window; click the floating window receiving the video stream to return to the call page.

[0057] Alternatively, if Fig.10 As shown, after switching to other applications, a highlighted block will be displayed in the top status bar to remind the user that they are in remote assistance. Click the highlighted block in the status bar to return to the call page.

[0058] Fig.12 is a schematic diagram of the end of the remote assistance provided by the present invention, such as Figure 8 and Fig.12 As shown, after the remote assistance connection has been established, click the remote assistance component again to end the remote assistance. After the end, the original camera image will be restored in the floating window.

[0059] The screen video sharing method provided by the embodiment of the present invention displays the video through a floating window when the user switches to other applications, and supports dragging and retracting the floating window to facilitate user operation, and can display a complete application interface to facilitate video customer service to locate problems.

[0060] In an optional embodiment, after receiving the remote assistance initiation instruction input by the user, the method further includes: Based on the screen parameters of the terminal device, generate fmtp parameters, wherein the fmtp parameters include a screen proportional scaling resolution of the terminal device; Sending a session initiation request to the video customer service, wherein the session initiation request includes the fmtp parameter; After receiving the response information from the video customer service, the original screen resolution video is scaled proportionally, and the scaled video stream is sent to the video customer service, the response information indicates that the session initiation request has been successfully processed, and the response information includes a resolution that is the same as the screen proportional scaling resolution.

[0061] Fig.13 FIG. 1 is a flow chart of video stream switching provided by the present invention, such as Fig.13 As shown in the figure, when the video sources of the camera screen and the remote assistance screen are switched, the user terminal will re-initiate resolution negotiation and generate corresponding negotiation parameters according to the current terminal screen resolution. Specifically, after the user selects remote assistance, the user terminal device dynamically generates fmtp (Format Parameters, which is an attribute used to describe the detailed parameters of the media encoding format) parameters according to the screen parameters, and resends the INVITE signaling, carrying fmtp in the SDP. After receiving the INVITE, the video customer service generates the corresponding fmtp parameters of the same resolution and replies 200OK.

[0062] Optionally, after receiving the remote auxiliary video, the video customer service detects whether the resolution has changed. After finding that the resolution has changed, the video rendering window is dynamically adjusted according to the new resolution to be consistent with the width and height of the remote service video stream to ensure normal display of the remote auxiliary screen.

[0063] The screen video sharing method provided by the embodiment of the present invention adds a resolution re-negotiation mechanism before remote assistance, so that the user terminal dynamically adjusts the screen aspect ratio to a video with the same aspect ratio suitable for transmission over the communication network. At the same time, after the video customer service receives the remote auxiliary video, it dynamically adjusts the size of the rendered video frame so that the remote auxiliary video is displayed on the desktop in proportion, ultimately solving the problems of black edges, incomplete display, compression deformation, etc. caused by different terminal resolutions.

[0064] Figure 2 This is the second flow chart of the screen video sharing method provided by the present invention. Figure 2 The preferred screen video sharing method process of the present invention is described.

[0065] Terminal device initiates call: When a user encounters a problem with the terminal device itself, he or she can use a terminal device that supports VoLTE video calls to initiate a call request to the video customer service. The establishment of the call depends on the SIP communication protocol.

[0066] Each terminal notifies whether it supports remote assistance: During the call establishment process, the SIP protocol carries identification field information to allow each terminal to notify whether its native call application supports remote assistance, screen sharing or other functions. This not only avoids functional abnormalities due to some terminals not supporting remote assistance, but also solves the problem of complex interaction logic and page operations caused by multiple signaling interactions.

[0067] Upgrade to two-way video call: The video customer service will upgrade the call with the user to a two-way video call. At this time, the remote assistance component on the native call application of the user's terminal device will change from gray (or hidden) to highlighted.

[0068] Remote assistance initiated by the terminal device: In a two-way video conversation, the user can initiate remote assistance through the remote assistance component on the native call application of the terminal device; because remote assistance will share the screen with the other party, there is a security risk. After the user initiates remote assistance, a prompt to confirm the remote assistance will pop up to avoid user misoperation.

[0069] The terminal device switches the video source: After the user confirms remote assistance, the terminal device switches the video source and switches the video originally captured by the front / rear camera to the screenshot video stream of the terminal screen. At this time, both parties can display the user's terminal screen.

[0070] Screen display effect adaptation: The shared screen is adapted through resolution re-negotiation to solve problems such as black edges, incomplete display, compression deformation, etc. caused by differences in shape, size, bit rate, resolution, etc. between the two terminals after screen sharing.

[0071] In summary, the screen video sharing method provided by the present invention reuses the SIP communication protocol when the call is established, and carries an identification field in the SIP protocol to allow each terminal to inform whether its own native call application supports remote assistance, screen sharing or other functions, which not only avoids functional abnormalities in some terminals due to not supporting remote assistance, but also solves the problems of complex interaction logic and complex page operations caused by multiple signaling interactions. In addition, the identification field carried in the SIP protocol can be extended to other multi-party conference functions of the terminal native call application; by adding a resolution re-negotiation mechanism before remote assistance, the user terminal is dynamically adjusted to a video with the same aspect ratio suitable for transmission over the communication network according to the screen aspect ratio. At the same time, after the video customer service receives the remote assistance video, it dynamically adjusts the size of the rendered video frame so that the remote assistance video is displayed on the desktop in proportion, ultimately solving the problems of black edges, incomplete display, compression deformation, etc. caused by different resolutions of different terminals; an optimization solution is provided for the operation experience of the remote assistance function.

[0072] The following is a description of a screen video sharing device provided by an embodiment of the present invention. The screen video sharing device described below and the screen video sharing method described above can be referenced to each other.

[0073] Fig.14 is a schematic diagram of the structure of the screen video sharing device provided by the present invention, such as Fig.14 As shown, the screen video sharing device may include but is not limited to; The call establishment module 1410 is used to initiate a call request to the video customer service through the SIP communication protocol, or accept a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; The video source setting module 1420 is used to: if the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, set the video source of the call to the screenshot video stream of the terminal device, so as to share the screenshot of the terminal device to the video customer service in real time. The remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0074] It should be noted that the screen video sharing device provided in the embodiment of the present invention can execute the screen video sharing method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0075] Fig.15 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.15 As shown, the electronic device may include: a processor 1510, a communications interface 1520, a memory 1530 and a communication bus 1540, wherein the processor 1510, the communications interface 1520 and the memory 1530 communicate with each other through the communication bus 1540. The processor 1510 may call the logic instructions in the memory 1530 to execute the screen video sharing method, the method comprising: initiating a call request to a video customer service through a SIP communication protocol, or accepting a call request initiated by the video customer service through a SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; If the identification field information indicates that the terminal device supports remote assistance, then upon receiving a remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device, and the remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0076] In addition, the logic instructions in the above-mentioned memory 1530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0077] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the screen video sharing method provided by the above methods, the method comprising: initiating a call request to a video customer service through a SIP communication protocol, or accepting a call request initiated by the video customer service through a SIP communication protocol, the SIP communication protocol includes identification field information, the identification field information indicates whether the terminal device supports remote assistance; If the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device to share the screenshot of the terminal device to the video customer service in real time. The remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0078] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the screen video sharing method provided by the above methods is implemented, the method comprising: initiating a call request to a video customer service through a SIP communication protocol, or accepting a call request initiated by the video customer service through a SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; If the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device to share the screenshot of the terminal device to the video customer service in real time. The remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

[0079] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0080] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A screen video sharing method, applied to a terminal device, characterized in that: include: Initiating a call request to the video customer service through the SIP communication protocol, or accepting a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; If the identification field information indicates that the terminal device supports remote assistance, then after receiving the remote assistance initiation instruction input by the user, the video source of the call is set to the screenshot video stream of the terminal device to share the screenshot of the terminal device to the video customer service in real time. The remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

2. The screen video sharing method according to claim 1, characterized in that: After initiating a call request to the video customer service through the SIP communication protocol, or accepting a call request initiated by the video customer service through the SIP communication protocol, the method further includes: During an audio call or a one-way video call, graying out the remote assistance component or hiding the remote assistance component; During a two-way video call, highlighting the remote assistance component; The remote assistance initiation instruction is an operation instruction for the remote assistance component.

3. The screen video sharing method according to claim 1 or 2, characterized in that: After receiving the remote assistance initiation instruction input by the user, setting the video source of the call to the screenshot video stream of the terminal device includes: After receiving the remote assistance initiation instruction input by the user, a remote assistance confirmation prompt box is displayed; After receiving the remote assistance confirmation initiation instruction input by the user through the remote assistance confirmation prompt box, a remote assistance connection is established and the video source of the call is set to the screenshot video stream of the terminal device. The remote assistance confirmation initiation instruction is an instruction to confirm the initiation of remote assistance.

4. The screen video sharing method according to claim 1 or 2, characterized in that: After establishing the remote auxiliary connection and setting the video source of the call to the screenshot video stream of the terminal device, the method further includes: hiding a floating window for receiving a video stream, wherein the received video stream is a video stream received from the video customer service; The call timing duration and remote assistance timing duration are displayed in a carousel based on the preset time interval.

5. The screen video sharing method according to claim 4, characterized in that: After establishing the remote auxiliary connection and setting the video source of the call to the screenshot video stream of the terminal device, the method further includes: After receiving an application switching instruction input by a user, displaying a floating window for receiving the video stream, wherein the application switching instruction is an instruction for instructing the terminal device to switch from a call page to another application page; In response to the user's dragging operation, the floating window for receiving the video stream is collapsed, and a floating window folding icon is displayed at the edge of the screen; In response to a click operation on the floating window folding icon, the floating window folding icon is collapsed, and the floating window for receiving the video stream is displayed.

6. The screen video sharing method according to claim 1 or 2, characterized in that: After receiving the remote assistance initiation instruction input by the user, the method further includes: Based on the screen parameters of the terminal device, generate fmtp parameters, wherein the fmtp parameters include a screen proportional scaling resolution of the terminal device; Sending a session initiation request to the video customer service, wherein the session initiation request includes the fmtp parameter; After receiving the response information from the video customer service, the original screen resolution video is scaled proportionally, and the scaled video stream is sent to the video customer service, the response information indicates that the session initiation request has been successfully processed, and the response information includes a resolution that is the same as the screen proportional scaling resolution.

7. A screen video sharing device, characterized in that: include: A call establishment module, used to: initiate a call request to the video customer service through the SIP communication protocol, or accept a call request initiated by the video customer service through the SIP communication protocol, wherein the SIP communication protocol includes identification field information, and the identification field information indicates whether the terminal device supports remote assistance; A video source setting module is used to: if the identification field information indicates that the terminal device supports remote assistance, then after receiving a remote assistance initiation instruction input by the user, set the video source of the call to the screenshot video stream of the terminal device, so as to share the screenshot of the terminal device to the video customer service in real time, and the remote assistance initiation instruction is an instruction instructing the terminal device to initiate remote assistance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the screen video sharing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the screen video sharing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the screen video sharing method according to any one of claims 1 to 6 is implemented.

Images