Screen sharing method, related electronic equipment and system

Abstract

The invention discloses a screen sharing method, related electronic equipment and a system, which can support a first terminal and a second terminal to share screen interface content with a video call application on the first terminal under a multi-screen cooperation function, and can support the second terminal to share the screen interface content with the video call application on the first terminal when a third terminal and the first terminal carry out video call. And the third terminal can watch the interface content shared by the second terminal through the video call application. By implementing the method provided by the invention, the second terminal can quickly share the screen interface content to the first terminal and the third terminal, the steps are simple, quick and efficient, and the office efficiency is further improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method for cross-device collaborative screen sharing, related electronic equipment and system. Background Art

[0002] Current video conferencing software can already support multiple participants sharing screens, but the screen sharing function depends on the conferencing software and the participating terminal, that is, the terminal itself needs to install the conferencing software and use the conference identity document (ID) to join the conference, and the conferencing software must provide a screen sharing function based on screen recording. For users who do not have or cannot install the conferencing software on their terminals but still want to join the conference and share the screen, this function has not been solved. Summary of the invention

[0003] The present application provides a method for sharing a screen, related electronic equipment and system, which can support a first terminal and a second terminal under a multi-screen collaboration function, where the second terminal shares the screen interface content to a video call application on the first terminal, and when a third terminal is in a video call with the first terminal, the third terminal can watch the interface content shared by the second terminal through the video call application.

[0004] In a first aspect, an embodiment of the present application provides a method for sharing a screen, which may include: a first terminal establishes a first connection with a second terminal. The first terminal obtains a first media stream of the second terminal, the first media stream includes audio and video data generated by the second terminal, the audio includes the sound collected by the microphone of the second terminal or the audio corresponding to the media file played by the second terminal, and the video includes the screen image recorded by the second terminal. The first terminal generates a second media stream based on the first media stream, and the second media stream includes the screen image recorded by the second terminal. The first terminal sends the second media stream to the third terminal through the second connection. The third terminal plays the second media stream received from the first terminal. The first terminal plays the third media stream received from the third terminal, and the third media stream includes the image collected by the camera of the third terminal or the sound collected by the microphone of the third terminal.

[0005] By implementing the method of the first aspect, the second terminal can quickly share the screen interface content with the first terminal and the third terminal, with simple steps, fast and efficient, further improving office efficiency. For example, the first terminal and the second terminal can support the multi-screen collaboration function, where the second terminal shares the screen interface content with the video call application on the first terminal, and when the third terminal is in a video call with the first terminal, the third terminal can watch the interface content shared by the second terminal through the video call application.

[0006] The display content of the media stream (including audio stream and video stream) sent by the first terminal to the third terminal includes but is not limited to the following three situations:

[0007] 1. The displayed content is edited by the first terminal.

[0008] After the first terminal receives the video stream collected from the second terminal, the user can make some control operations on the first terminal side, edit, set or adjust the video stream, generate a new video stream, and then send it to the third terminal. For example, the user can make some annotations on the video screen of the first terminal, and then these annotations are combined with the original video screen to form a new video screen and send it to the third terminal. For another example, the user can set the video screen sent by the first terminal to the third terminal to include not only the screen recording screen of the second terminal, but also the user's avatar collected by the camera of the first terminal. In this way, the third terminal side not only displays the screen recording screen from the second terminal, but also displays the avatar of the user of the first terminal, and the user's video call experience is better. In addition, the picture parameters can also be adjusted on the video call interface of the first terminal. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc.

[0009] In this way, the first terminal can adjust the image captured by the second terminal using its own editing capabilities, generate a new display screen and send it to the third terminal, which increases the convenience of use for the user of the first terminal.

[0010] 2. The displayed content is edited by the second terminal.

[0011] After the second terminal generates the screen recording, the user can perform some control operations on the second terminal to edit, set or adjust the video stream, generate a new video stream, and then send it to the first terminal.

[0012] For example, the user can make some annotations on the screen recording of the second terminal, and then these annotations together with the original screen recording are synthesized into a new video screen and sent to the first terminal, and then sent by the first terminal to the third terminal. For another example, the user can set the video screen sent by the second terminal to the first terminal to include not only the screen recording of the second terminal, but also the user's avatar captured by the camera of the second terminal. In this way, the first terminal side and the third terminal side not only display the screen recording from the second terminal, but also display the avatar of the user of the second terminal, so that the user's video call experience is better. In addition, the picture parameters can also be adjusted on the user interface of the second terminal. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc.

[0013] In this way, the second terminal can use its own editing capabilities to adjust the images it has collected, generate a new display screen and send it to the first terminal, which adds convenience to the user of the second terminal.

[0014] 3. The displayed content is collaboratively edited by the first terminal and the second terminal.

[0015] In combination with the description of the first two aspects, in one implementation, both the first terminal and the second terminal can have the function of editing the video screen. In this way, the first terminal and the second terminal can collaboratively complete the editing of the video screen, as close as possible to the display effect that the user wants to obtain, which adds convenience to the users of the first terminal and the second terminal, and avoids the situation where the editing display effect cannot satisfy both parties when the first terminal or the second terminal edits the display screen separately.

[0016] For example, the first terminal can edit the edited screen again based on the editing of the screen recording screen by the second terminal. After the first terminal or the second terminal edits the screen recording screen, the display screen is synchronized on the display interface of the first terminal and the second terminal.

[0017] Furthermore, after the first terminal sends an instruction to adjust the display effect to the second terminal, the second terminal can also perform compatibility processing on the instruction so that the second terminal can respond to the processed instruction and effectively complete the adjustment of the corresponding display effect. This is because the device types and device models of the first terminal and the second terminal may not be the same. When the first terminal responds to the user's operation of adjusting the display effect, some instructions that the second terminal does not support or cannot recognize may be generated, and the second terminal may not be able to directly respond to these instructions to adjust the display effect of the interface. The second terminal's compatibility processing of the instruction enables the second terminal to respond to the user's operation on the first terminal, and realizes the first terminal's control over the second terminal's adjustment of the display interface effect.

[0018] From the above situation, it can be seen that not only the second terminal can have the ability to edit the screen recording screen or other video streams, but the first terminal can also have the ability to edit the screen recording screen or other video streams from the second terminal. This strengthens the control effect of the first terminal over the display interface of the second terminal and improves the user experience.

[0019] In an embodiment of the present application, the first terminal displays the image of the second terminal by using the data stream channel collected by its own camera to transmit the image from the second terminal, that is, the first terminal originally uses the data stream channel to transmit the data of the image collected by its own camera, but in the scenario of the embodiment of the present application, the data stream channel is used to transmit the screen recording data stream or other video stream from the second terminal. In this way, the upper-layer application of the first terminal does not need to perceive whether the displayed image comes from the first terminal or the second terminal. The video stream collected by the second terminal is equivalent to the video stream collected by a remote camera of the first terminal. There is no need to modify the way the third-party application calls the system interface, that is, there is no need for the third-party application to adapt. The first terminal can send the acquired image of the second terminal to the upper-layer application, which expands the use scenario of screen sharing between the first terminal and the second terminal.

[0020] In combination with the first aspect, in an implementation manner, before the first terminal sends the second media stream to the third terminal through the second connection, the first terminal establishes a second connection with the third terminal.

[0021] In combination with the first aspect, in one implementation, before the first terminal establishes a first connection with the second terminal, the first terminal establishes a second connection with the third terminal. The first terminal plays a third media stream received from the third terminal, where the third media stream includes a picture captured by a camera of the third terminal or a sound captured by a microphone of the third terminal. The third terminal plays a fourth media stream received from the first terminal, where the fourth media stream includes a picture captured by the camera of the first terminal or a sound captured by the microphone of the first terminal.

[0022] In combination with the first aspect, in an implementation manner, before the first terminal acquires the first media stream of the second terminal, the first terminal detects a first user operation, where the first user operation is used to instruct the second terminal to start recording a screen image.

[0023] In combination with the first aspect, in an implementation manner, the second media stream is the same as the first media stream.

[0024] In combination with the first aspect, in one implementation, the second media stream further includes a picture captured by a camera of the first terminal.

[0025] In combination with the first aspect, in an implementation manner, the second media stream further includes an editing trace on a screen image recorded by the first terminal at the second terminal.

[0026] In combination with the first aspect, in an implementation manner, the second media stream further includes sound collected by a microphone of the first terminal.

[0027] In combination with the first aspect, in one implementation, the screen image recorded by the second terminal includes: the entire screen image recorded by the second terminal, or the screen image corresponding to a partial area recorded by the second terminal, or the screen image corresponding to a specified file recorded by the second terminal.

[0028] In combination with the first aspect, in one implementation, the first terminal transmits the video stream included in the second media stream to an image stream channel for capturing images through a camera of the first terminal, the first terminal transmits the audio stream included in the second media stream to an audio stream channel for capturing sounds through a microphone of the first terminal, and the first terminal sends the media streams in the image stream channel and the audio stream channel to a third terminal through a second connection.

[0029] In combination with the first aspect, in one implementation, after the first terminal establishes a first connection with the second terminal, the first terminal negotiates a first parameter with the second terminal, and the first parameter includes one or more of the following: the bit rate of screen recording or audio recording, the resolution of screen recording, the sound source of recording, the cache path, the audio or video encoding and decoding method, the specified area of ​​the recording screen, the specified recorded file, the rotation strategy of the video window, and the sound source of the recording includes the audio corresponding to the media file played by the second terminal, or the audio collected by the microphone of the second terminal.

[0030] In combination with the first aspect, in one implementation, the first user interface of the first terminal includes a first window and a second window, and the second user interface of the third terminal includes a third window and a fourth window.

[0031] In combination with the first aspect, in one implementation method, after the first terminal sends a second media stream to a third terminal through a second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0032] In combination with the first aspect, in one implementation method, after the first terminal sends a second media stream to a third terminal through a second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0033] In combination with the first aspect, in one implementation, the first user interface of the first terminal further includes a fifth window, and the second user interface of the third terminal further includes a sixth window. The picture displayed in the fifth window is the picture captured by the camera of the first terminal, and the picture displayed in the sixth window is the picture captured by the camera of the first terminal.

[0034] In combination with the first aspect, in one implementation, when the first terminal plays a third media stream received from a third terminal, the picture displayed in the first window is the picture captured by the camera of the third terminal, and the picture displayed in the second window is the picture captured by the camera of the first terminal. When the third terminal plays a fourth media stream received from the first terminal, the picture displayed in the third window is the picture captured by the camera of the first terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0035] In a second aspect, the present application provides a method for sharing a screen, which may include: a first terminal establishes a first connection with a second terminal. The first terminal obtains a first media stream from the second terminal, the first media stream includes audio and video data generated by the second terminal, the audio includes the sound collected by the microphone of the second terminal or the audio corresponding to the media file played by the second terminal, and the video includes the screen image recorded by the second terminal. The first terminal generates a second media stream based on the first media stream, and the second media stream includes the screen image recorded by the second terminal. The first terminal sends a second media stream to a third terminal through a second connection, and the screen image recorded by the second terminal included in the second media stream is displayed on the screen of the third terminal. The first terminal plays a third media stream received from the third terminal, and the third media stream includes the image collected by the camera of the third terminal or the sound collected by the microphone of the third terminal.

[0036] By implementing the method of the second aspect, the second terminal can quickly share the screen interface content with the first terminal and the third terminal, with simple steps, fast and efficient, further improving office efficiency. For example, the first terminal and the second terminal can support the multi-screen collaboration function, and the second terminal shares the screen interface content with the video call application on the first terminal. When the third terminal is in a video call with the first terminal, the third terminal can watch the interface content shared by the second terminal through the video call application.

[0037] In conjunction with the second aspect, in an implementation manner, before the first terminal sends the second media stream to the third terminal through the second connection, the first terminal establishes a second connection with the third terminal.

[0038] In combination with the second aspect, in one implementation, before the first terminal establishes a first connection with the second terminal, the first terminal establishes a second connection with the third terminal. The first terminal sends a fourth media stream to the third terminal through the second connection, and the fourth media stream includes a picture captured by the camera of the first terminal or a sound captured by the microphone of the first terminal, and the picture captured by the camera of the first terminal is displayed on the screen of the third terminal. The first terminal receives a third media stream sent by the third terminal through the second connection. The first terminal plays a third media stream received from the third terminal, and the third media stream includes a picture captured by the camera of the third terminal or a sound captured by the microphone of the third terminal.

[0039] In combination with the second aspect, in an implementation manner, before the first terminal acquires the first media stream of the second terminal, the first terminal detects a first user operation, where the first user operation is used to instruct the second terminal to start recording a screen image.

[0040] In conjunction with the second aspect, in one implementation, the second media stream is the same as the first media stream.

[0041] In conjunction with the second aspect, in one implementation, the second media stream further includes a picture captured by a camera of the first terminal.

[0042] In conjunction with the second aspect, in one implementation, the second media stream further includes an editing trace on a screen image recorded by the first terminal at the second terminal.

[0043] In conjunction with the second aspect, in one implementation, the second media stream further includes sound collected by a microphone of the first terminal.

[0044] In conjunction with the second aspect, in one implementation, the screen image recorded by the second terminal includes: the entire screen image recorded by the second terminal, or the screen image corresponding to a partial area recorded by the second terminal, or the screen image corresponding to a specified file recorded by the second terminal.

[0045] In conjunction with the second aspect, in one implementation, the first terminal transmits the video stream included in the second media stream to an image stream channel for collecting images by a camera of the first terminal. The first terminal transmits the audio stream included in the second media stream to an audio stream channel for collecting sounds by a microphone of the first terminal. The first terminal sends the media streams in the image stream channel and the audio stream channel to the third terminal through the second connection.

[0046] In combination with the second aspect, in one implementation, after the first terminal establishes a first connection with the second terminal, the first terminal negotiates a first parameter with the second terminal, and the first parameter includes one or more of the following: the bit rate of screen recording or audio recording, the resolution of screen recording, the sound source of recording, the cache path, the audio or video encoding and decoding method, the specified area of ​​the recording screen, the specified recorded file, the rotation strategy of the video window, and the sound source of the recording includes the audio corresponding to the media file played by the second terminal, or the audio collected by the microphone of the second terminal.

[0047] In conjunction with the second aspect, in one implementation, the first user interface of the first terminal includes a first window and a second window, and the second user interface of the third terminal includes a third window and a fourth window.

[0048] In combination with the second aspect, in one implementation method, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0049] In combination with the second aspect, in one implementation method, after the first terminal sends a second media stream to a third terminal through a second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0050] In conjunction with the second aspect, in one implementation, the first user interface of the first terminal further includes a fifth window, and the second user interface of the third terminal further includes a sixth window. The picture displayed in the fifth window is the picture captured by the camera of the first terminal, and the picture displayed in the sixth window is the picture captured by the camera of the first terminal.

[0051] In conjunction with the second aspect, in one implementation, after the first terminal sends a fourth media stream to the third terminal through the second connection, the picture displayed in the third window is the picture captured by the camera of the first terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal. After the first terminal receives the third media stream sent by the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, and the picture displayed in the second window is the picture captured by the camera of the first terminal.

[0052] In a third aspect, the present application provides an electronic device, which may include: a communication device, a display screen, a memory and a processor coupled to the memory, multiple applications, and one or more programs; the display screen is used to display images, and the memory stores computer executable instructions, and when the processor executes the instructions, the electronic device realizes any function of the first terminal in the second aspect.

[0053] In a fourth aspect, the present application provides a communication system, which includes: a first terminal, a second terminal and a third terminal, wherein the first terminal can implement any function of the first terminal in the second aspect.

[0054] In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which a computer program is stored. The computer program includes executable instructions, which, when executed by a processor, enable the processor to perform operations corresponding to the method provided in the second aspect.

[0055] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on an electronic device, enables the electronic device to execute any possible implementation method in the second aspect.

[0056] In the seventh aspect, an embodiment of the present application provides a chip system, which can be applied to an electronic device. The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to implement any possible implementation method as in the second aspect.

[0057] By implementing the above method provided in the present application, the second terminal can quickly share the screen interface content with the first terminal and the third terminal. The steps are simple, fast and efficient, which further improves office efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a video conference scenario;

[0059] Figure 2 A schematic diagram of a communication system provided in an embodiment of the present application;

[0060] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0061] Figure 4 A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0062] Figure 5 A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0063] Figure 6A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0064] Fig. 7A A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0065] Figure 7B A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0066] Fig. 8A A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0067] Figure 8B A schematic diagram of a video call service scenario interface provided in an embodiment of the present application;

[0068] Fig.9A A schematic diagram of a user interface provided in an embodiment of the present application;

[0069] Fig. 9B A schematic diagram of a user interface provided in an embodiment of the present application;

[0070] Fig. 10A A schematic diagram of a user interface provided in an embodiment of the present application;

[0071] Fig. 10B A schematic diagram of a user interface provided in an embodiment of the present application;

[0072] Fig.11 A flowchart of a method for sharing a screen provided in an embodiment of the present application;

[0073] Fig.12 A flowchart of a method for sharing a screen provided in an embodiment of the present application;

[0074] Fig.13 A schematic diagram of software modules of a communication system provided in an embodiment of the present application;

[0075] Fig.14 A flowchart of a screen sharing method provided in an embodiment of the present application;

[0076] Fig.15 A schematic diagram of a screen sharing process provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0078] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

[0079] The term "user interface (UI)" in the following embodiments of the present application refers to the medium interface for interaction and information exchange between an application (APP) or an operating system (OS) and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that can be recognized by the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0080] In the current mainstream multi-person video conferencing scenarios, joining a video conference requires the terminal itself to have the conference software installed and use the conference ID to join the conference. When participating in a video conference, the conference software will provide a screen sharing function based on screen recording, which makes it convenient for participants to share and watch the conference content.

[0081] like Figure 1 In the video conferencing scenario shown, both mobile phone A and mobile phone B have conference software A installed, and can enter the conference by logging in to the same conference ID.

[0082] exist Figure 1In the conference, mobile phone A can share its desktop or files based on the screen recording function, and then the video stream of the screen recorded by mobile phone A can be transmitted to mobile phone B via the Internet. Mobile phone B can play the received video stream of the screen recorded by mobile phone A. In this way, mobile phone A and mobile phone B, as well as other terminals connected to the meeting, can conduct video conferences and communicate in the meeting online in real time.

[0083] Due to factors such as development capabilities and progress, some conferencing software does not support all types of terminals or all types of operating systems.

[0084] For example, conference software A only has a mobile client, but no personal computer (PC) client, so computer A cannot install conference software A, nor can it join the conference to share the screen. If the conference presentation data is stored on computer A, but computer A cannot join the conference to share the screen, computer A needs to send the conference presentation data to a terminal that can share the screen, such as mobile phone A, and then perform the presentation, which is very inconvenient for users.

[0085] For terminals that do not have or cannot install conference software, if you want to join a conference and share the screen, you can also use a browser application. Users can enter the conference link corresponding to the conference ID in the browser application, log in to the account, and enter the conference. The process is relatively cumbersome. The essence of joining a conference through conference software or browser applications is the same, both use the media on the terminal that can carry the conference application to conduct video conferencing.

[0086] For another example, two people are discussing work through video call software (such as Changlian Call, WeChat), and at this time they need to demonstrate the materials on the PC. The solution may be to send the materials to the other party through email or communication software platforms, or to reschedule a network meeting and share the materials after both parties join the meeting, but these solutions are cumbersome for users and the sharing efficiency is low.

[0087] The present application provides a method for sharing a screen, which can be applied to the related electronic devices and systems provided in the present application. The electronic devices may include mobile phones, tablet computers (portable android devices, PADs), personal computers (personal computers, PCs), smart TVs, electronic conference whiteboards and other terminal devices, and the present application does not impose any restrictions on the specific types of electronic devices.

[0088] The multi-screen collaboration function refers to a distributed communication technology that can realize cross-operating system and cross-device collaborative work. For example, after a mobile phone is connected to a PC, the mobile phone and the PC can share resources and collaborate to improve event processing efficiency.

[0089] The screen sharing method provided in the present application can support the first terminal and the second terminal under the multi-screen collaboration function, where the second terminal shares the screen interface content (such as the recorded screen image) to the video call APP on the first terminal. When the third terminal makes a video call with the first terminal, the third terminal can watch the interface content shared by the second terminal through the video call APP without going through the conference application.

[0090] The display content of the media stream (including audio stream and video stream) sent by the first terminal to the third terminal includes but is not limited to the following three situations:

[0091] 1. The displayed content is edited by the first terminal.

[0092] After the first terminal receives the video stream collected by the second terminal, the user can perform some control operations on the first terminal to edit, set or adjust the video stream, generate a new video stream, and then send it to the third terminal.

[0093] For example, the user can make some annotations on the video screen of the first terminal, and then these annotations are combined with the original video screen to form a new video screen, which is sent to the third terminal.

[0094] For example, the user can set the video image sent by the first terminal to the third terminal to include not only the screen recording image of the second terminal, but also the user's portrait captured by the camera of the first terminal. In this way, the third terminal side not only displays the screen recording image from the second terminal, but also displays the portrait of the user of the first terminal, providing the user with a better video call experience.

[0095] Furthermore, the picture parameters can be adjusted on the video call interface of the first terminal. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc. Among them, the image cropping size may be 3:3, 3:4, 9:16, etc.; the image color temperature calibration mode may be daylight, fluorescent, incandescent, shadow, cloudy calibration mode, etc.; the beauty / body type may be face slimming, body slimming, skin grinding, whitening, big eyes, acne removal, etc.; the filter type may be Japanese, texture, bright, soft light, cyberpunk, etc.; the stickers may be stickers such as expressions, animals, scenery, illustrations, etc.

[0096] In this way, the first terminal can adjust the image captured by the second terminal using its own editing capabilities, generate a new display screen and send it to the third terminal, which increases the convenience of use for the user of the first terminal.

[0097] 2. The displayed content is edited by the second terminal.

[0098] After the second terminal generates the screen recording, the user can perform some control operations on the second terminal to edit, set or adjust the video stream, generate a new video stream, and then send it to the first terminal.

[0099] For example, the user can make some annotations on the screen recording of the second terminal, and then these annotations are combined with the original screen recording to form a new video screen that is sent to the first terminal, and then sent by the first terminal to the third terminal.

[0100] For example, the user can set the video image sent by the second terminal to the first terminal to include not only the screen recording image of the second terminal, but also the user's portrait captured by the camera of the second terminal. In this way, the first terminal side and the third terminal side not only display the screen recording image from the second terminal, but also display the portrait of the user of the second terminal, thereby improving the user's video call experience.

[0101] Furthermore, the user interface of the second terminal can also adjust the picture parameters. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc.

[0102] In this way, the second terminal can use its own editing capabilities to adjust the images it has collected, generate a new display screen and send it to the first terminal, which adds convenience to the user of the second terminal.

[0103] 3. The displayed content is collaboratively edited by the first terminal and the second terminal.

[0104] In combination with the description of the first two aspects, in one implementation, both the first terminal and the second terminal can have the function of editing the video screen. In this way, the first terminal and the second terminal can collaboratively complete the editing of the video screen, as close as possible to the display effect that the user wants to obtain, which adds convenience to the users of the first terminal and the second terminal, and avoids the situation where the editing display effect cannot satisfy both parties when the first terminal or the second terminal edits the display screen separately.

[0105] For example, the first terminal can edit the edited screen again based on the editing of the screen recording screen by the second terminal. After the first terminal or the second terminal edits the screen recording screen, the display screen is synchronized on the display interface of the first terminal and the second terminal.

[0106] Furthermore, after the first terminal sends an instruction for adjusting the display effect to the second terminal, the second terminal may also perform compatibility processing on the instruction, so that the second terminal can effectively complete the adjustment of the corresponding display effect in response to the processed instruction.

[0107] This is because the device types and device models of the first terminal and the second terminal may be different. When the first terminal responds to the user's operation of adjusting the display effect, some instructions that the second terminal does not support or cannot recognize may be generated. The second terminal may not be able to directly respond to these instructions to adjust the display effect of the interface.

[0108] The second terminal performs compatibility processing on the instruction so that the second terminal can respond to the user's operation on the first terminal, thereby realizing the first terminal's control over the second terminal to adjust the display interface effect.

[0109] From the above situation, it can be seen that not only the second terminal can have the ability to edit the screen recording screen or other video streams, but the first terminal can also have the ability to edit the screen recording screen or other video streams from the second terminal. This strengthens the control effect of the first terminal over the display interface of the second terminal and improves the user experience.

[0110] In an embodiment of the present application, the first terminal displays the image of the second terminal by using the data stream channel collected by its own camera to transmit the image from the second terminal, that is, the first terminal originally uses the data stream channel to transmit the data of the image collected by its own camera, but in the scenario of the embodiment of the present application, the data stream channel is used to transmit the screen recording data stream or other video stream from the second terminal. In this way, the upper-layer application of the first terminal does not need to perceive whether the displayed image comes from the first terminal or the second terminal. The video stream collected by the second terminal is equivalent to the video stream collected by a remote camera of the first terminal. There is no need to modify the way the third-party application calls the system interface, that is, there is no need for the third-party application to adapt. The first terminal can send the acquired image of the second terminal to the upper-layer application, which expands the use scenario of screen sharing between the first terminal and the second terminal.

[0111] By implementing the method provided in the present application, the second terminal can quickly share the screen interface content with the first terminal and the third terminal. The steps are simple, fast and efficient, which further improves office efficiency.

[0112] For example, in the aforementioned mainstream multi-person video conferencing scenario, participation in the video conference depends on the terminal itself having the conference software installed, and the conference software also needs to provide a screen sharing function based on screen recording.

[0113] In the technical solution provided in the present application, assuming that mobile phone A and mobile phone B are holding a meeting through video conferencing software or video call software, and the meeting materials that need to be demonstrated are on computer A, then computer A does not need to install video conferencing software or video call software, nor does mobile phone A need to support the screen sharing function of screen recording. When mobile phone A is connected to computer A, based on the multi-screen collaboration function, computer A can inject the recorded demonstration content into the camera of mobile phone A, and mobile phone B can watch the meeting materials shared from the screen of computer A through the video stream received from the camera of mobile phone A.

[0114] The technical solution provided by this application can enable users to realize the screen sharing function across devices on an APP that only supports video calls, and it does not rely on the installation of conference software on the terminal.

[0115] First, the communication system provided by the embodiment of the present application is introduced.

[0116] Figure 2 A communication system 10 provided in an embodiment of the present application is shown.

[0117] The communication system 10 may include multiple intelligent terminal devices, and communication connections are established between these multiple terminal devices. The terminal device may be a smart phone, a laptop computer, a desktop computer, a tablet computer, an electronic conference whiteboard, a smart screen (smart TV) or other types of electronic devices, such as desktop computers, laptop computers, handheld computers, augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, car machines, game consoles, other smart wearable devices, etc. This application does not impose any restrictions on this. In this embodiment, the terminal device may also be referred to as a terminal, and the terminal device is generally an intelligent electronic device that can provide a user interface, interact with a user, and provide business functions to the user.

[0118] Each terminal device in the communication system 10 can be equipped with system, system, system, system (HarmonyOS, HOS) or other types of operating systems. The operating systems of the various terminal devices in the communication system 10 may be the same or different, and this application does not limit this. In some embodiments, multiple terminals are equipped with system, then the system consisting of multiple terminals can be called Super virtual device, also called A super terminal refers to the integration of the capabilities of multiple terminals through distributed technology, which are stored in a virtual hardware resource pool. The terminal capabilities are uniformly managed, scheduled and integrated according to business needs to provide services to the outside world, enabling fast connection, mutual assistance of capabilities and resource sharing between different terminals.

[0119] like Figure 2 As shown in , the communication system 10 includes a first terminal, a second terminal and a third terminal. Each terminal may be configured with a camera.

[0120] A first connection is established between the first terminal and the second terminal, and a second connection is established between the first terminal and the third terminal. The embodiment of the present application does not limit the types of the first connection and the second connection, and the first connection or the second connection can be a wireless connection or a wired connection.

[0121] In some embodiments, the second connection may be an Internet connection, and the first terminal and the third terminal may be connected and communicate via the Internet, such as making a video call. The first terminal and the second terminal may access the Internet via a wireless communication network (such as a wireless fidelity (Wi-Fi) network) or a mobile communication network (such as a mobile communication 2G / 3G / 4G / 5G network).

[0122] The second terminal can record the desktop and then send the recorded video stream to the first terminal through the first connection. The first connection can include a wired connection, such as a universal serial bus (USB) connection, or a wireless connection, such as a Bluetooth (BT) connection, a wireless fidelity (Wi-Fi) connection, a hotspot connection, etc., to enable communication between the first terminal and the second terminal in the case of the same account, no account, or different accounts.

[0123] In some embodiments, multiple terminals can log in to the same account to connect and communicate through the Internet. Multiple terminals can also log in to different accounts, but connect by binding. For example, the first terminal and the second terminal can log in to different accounts, and the first terminal binds the second terminal to itself in the device management application, and then connects through the device management application.

[0124] The embodiment of the present application does not limit the type of communication connection. Data can be transmitted and interacted between various terminals through various communication connection types. In addition, the terminals can also be connected and communicated in combination with any of the above methods, and the embodiment of the present application does not limit this.

[0125] The terminal device may be configured with a mobile communication module and a wireless communication module.

[0126] The mobile communication module can provide wireless communication solutions including 2G / 3G / 4G / 5G applied on the terminal.

[0127] The wireless communication module may include a Bluetooth (BT) module and / or a wireless local area network (WLAN) module, etc.

[0128] Among them, the Bluetooth module can provide solutions for one or more Bluetooth communications including classic Bluetooth (Bluetooth 2.1) or Bluetooth low energy (BLE), and the WLAN module can provide solutions for one or more WLAN communications including wireless fidelity peer-to-peer (Wi-Fi P2P), wireless fidelity local area networks (Wi-Fi LAN) or wireless fidelity software access point (Wi-Fi softAP).

[0129] In some embodiments, Wi-Fi P2P refers to allowing devices in a wireless network to connect to each other in a peer-to-peer manner without going through a wireless router. The system is also called wireless fidelity direct (Wi-Fi direct). Devices that establish a Wi-Fi P2P connection can exchange data directly through Wi-Fi (must be in the same frequency band) without connecting to a network or hotspot, achieving point-to-point communication, such as transferring files, pictures, videos and other data. Compared with Bluetooth, Wi-Fi P2P has the advantages of faster search and transmission speeds and longer transmission distances.

[0130] The first terminal can replace the video stream shot by its own camera with the recorded video stream from the second terminal. In this way, when the first terminal and the third terminal have a video call, the third terminal interface displays the recorded video stream from the second terminal instead of the picture shot by the camera of the first terminal. In this way, the second terminal can share the screen and participate in the video conference without complicated steps.

[0131] It should be noted that Figure 2The communication system 10 shown is only used to assist in describing the technical solution provided by the embodiment of the present application, and does not constitute a limitation on the embodiment of the present application. In actual business scenarios, the communication system 10 may include more or fewer terminal devices, and the present application does not impose any limitation on the number of terminals, connection methods, etc.

[0132] The following is an exemplary description of the structure of the electronic device provided in the embodiments of the present application.

[0133] The electronic device provided in the embodiment of the present application can be Figure 2 Any electronic device in the communication system 10 shown may be the first terminal, the second terminal, the third terminal or other electronic devices described in the embodiment of the present application, and is not limited here. The electronic device 100 is used as an example for description.

[0134] In the embodiment of the present application, the electronic device 100 may include but is not limited to a mobile phone, a tablet computer, a smart screen (smart TV), an electronic conference whiteboard, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device, and / or a smart city device. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 100.

[0135] Figure 3 A schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0136] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0137] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0138] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0139] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0140] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0141] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0142] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0143] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0144] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0145] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0146] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0147] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0148] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0149] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0150] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0151] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0152] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0153] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0154] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0155] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0156] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0157] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0158] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0159] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0160] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0161] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0162] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0163] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0164] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0165] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0166] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0167] Random access memory has the characteristics of fast read / write speed and volatility. Volatility means that once the power is off, the data stored in the RAM will disappear. Generally, the static power consumption of random access memory is very low, and the operating power consumption is relatively large. The data in RAM is memory data, which can be read at any time and disappears when the power is off.

[0168] Non-volatile memory has the characteristics of non-volatility and stable storage data. Non-volatility means that the stored data will not disappear after power failure, and the data can be saved for a long time. The data in NVM includes application data, which can be stably stored in NVM for a long time. Application data refers to the content written during the operation of application programs or service processes, such as photos or videos obtained by camera applications, text edited by users in document applications, etc.

[0169] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0170] Non-volatile memory may include magnetic disk storage, flash memory, etc.

[0171] Disk storage devices are storage devices that use disks as storage media. They have the characteristics of large storage capacity, high data transfer rate, and long-term preservation of stored data.

[0172] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0173] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0174] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0175] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0176] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0177] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0178] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0179] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0180] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0181] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0182] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0183] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0184] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0185] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0186] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0187] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0188] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0189] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0190] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0191] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0192] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0193] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0194] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0195] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0196] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0197] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0198] The screen sharing method provided in the embodiment of the present application can be applied to various business scenarios, including but not limited to:

[0199] (1) Video conferencing scenario

[0200] For example, the first terminal and the third terminal are conducting a video conference through conference software, and the conference materials that the first terminal needs to demonstrate are on the second terminal. The first terminal can connect to the second terminal, and the second terminal sends its recorded screen video stream, that is, the conference materials that need to be demonstrated, to the first terminal, and the first terminal then sends the demonstrated conference materials to the third terminal. In this way, when the third terminal has not established a connection with the second terminal and the second terminal has not accessed the video conference through the conference software, the third terminal can display the demonstration content recorded by the second terminal.

[0201] (2) Video call scenario

[0202] For example, the first terminal and the third terminal are having a video chat through a video call software. During the chat, a certain topic is discussed, and information related to the topic is stored on the second terminal, such as work documents, children's photos, etc.

[0203] In order to facilitate the display to the other party and let the other party watch, the first terminal can be connected to the second terminal, and the second terminal sends its recorded screen video stream, that is, the picture or document to be displayed, to the first terminal, and the first terminal then sends the picture or document to be displayed to the third terminal.

[0204] In this way, when the third terminal is not connected to the second terminal, the third terminal can display the content recorded by the second terminal. When the first terminal is connected to the third terminal, the first terminal can quickly and conveniently display the content of the second terminal to the third terminal without complicated operations, giving the user a more convenient video call experience.

[0205] (3) Live broadcast scenario

[0206] In a live broadcast scenario, the first terminal and the second terminal are connected. Some demonstration materials are stored on the second terminal. The host can choose that the current live broadcast content is the image captured by the first terminal, or switch to the demonstration materials on the second terminal. The second terminal can send the video stream to the first terminal, and the first terminal can upload the video stream to the live broadcast server, which will be distributed by the live broadcast server to more users for viewing.

[0207] During this process, the anchor can control the switching of the currently displayed content to be the video stream captured by the camera of the first terminal or the video stream recorded by the second terminal.

[0208] The above described scenarios are only examples and do not constitute any limitation on other embodiments of the present application. Not limited to the above scenarios, the screen sharing method provided in the embodiments of the present application can be applied to other social scenarios, office scenarios, shopping scenarios, etc.

[0209] In conjunction with the description of the above communication system 10, reference Figure 4 , Figure 5 , Figure 6 , Fig. 7A , Figure 7B , Fig. 8A , Figure 8B , Figure 4-8B A video conferencing service scenario and a related user interface are exemplarily shown.

[0210] like Figure 4 As shown, user A and user B are having a video conference. User A uses mobile phone A, and user B uses mobile phone B. Mobile phone A and mobile phone B use a video call application to transmit video data through the Internet.

[0211] like Figure 4 As shown, the main window 401 of the video call user interface of mobile phone A displays the image of the other party, that is, user B, which is the image captured by the camera of mobile phone B. The main window 401 is a window covering the entire screen, and the main window 401 can display the video stream of mobile phone B received by mobile phone A, that is, the image of user B.

[0212] In the main window 401, there are also displayed a voice on / off control 402, a camera on / off control 403, a "leave meeting" control 404, a camera switch control 405, a floating window 406, etc. The floating window 406 can display the image of user A captured by the local camera of mobile phone A.

[0213] When the user clicks the switch camera control 405, the mobile phone A can switch the camera currently capturing images, for example, it can switch from the front camera to the rear camera to capture images.

[0214] The images displayed in the main window 401 and the floating window 406 can be swapped, that is, the main window 401 can display the image of user A, and the floating window 406 can display the image of user B. The method for swapping the images can be to switch the view size, that is, to swap the window size of the floating window 406 with that of the main window 401, or to switch the read video stream, that is, to swap the video stream obtained by the main window 401 with that obtained by the floating window 406.

[0215] In the embodiment of the present application, the main window and the floating window are distinguished according to the window properties (rather than according to the main body of the video stream), that is, the window that covers the entire screen to display the video image is called the main window, and the window that is superimposed on the main window is called the floating window. Generally speaking, the floating window covers the main window and is movable. The floating window is small in size and only occupies part of the display area of ​​the main window. The user can manually drag the display position of the floating window.

[0216] In one implementation, when the user clicks on the floating window 406 area, the main window 401 and the floating window 406 can swap the displayed images, that is, the original floating window 406 displaying the image of user A is switched to the main window 401 displaying the image of user A, and the original main window 401 displaying the image of user B is switched to the floating window 406 displaying the image of user B.

[0217] In the main window 407 of the video call user interface of the other end mobile phone B, an image of user A received through the Internet is displayed. The image of user A is captured by the camera of mobile phone A.

[0218] Similarly, the main window 407 also has controls for turning on / off voice, turning on / off camera, "leaving meeting" controls, switching camera controls, floating window 408, etc. The floating window 408 displays the image of user B captured by the local camera of mobile phone B.

[0219] Similarly, the floating window 408 covers the main window 407 and is movable. In one implementation, when the user clicks on the floating window 408 area, the main window 407 and the floating window 408 can swap display images, that is, the original floating window 408 displaying the image of user B is switched to the main window 407 displaying the image of user B, and the original main window 407 displaying the image of user A is switched to the floating window 408 displaying the image of user A.

[0220] In some embodiments, Figure 5 As shown, the conference materials that user A needs to present are on computer A, and mobile phone A can establish a Wi-Fi P2P connection with computer A. The connection between mobile phone A and computer A is not limited to Wi-Fi P2P connection. The connection between mobile phone A and computer A can also be other types of wireless connections or wired connections (such as USB connections), which are not listed here. The conference materials are displayed in the interface 501 of computer A. Computer A can record the interface 501, and then send the recorded media stream (including audio stream, video stream, etc.) to mobile phone A through the connection. Mobile phone A can send the recorded media stream received from computer A to mobile phone B through the video call application.

[0221] like Figure 5 As shown, in the main window 502 of the user interface of mobile phone A, the image of user B collected by mobile phone B and received through the video call application is displayed, and in the floating window 503, the screen of the recording interface 501 of computer A is displayed. Mobile phone B receives the media stream from mobile phone A through the video call application connection, and the main window 504 of the user interface of mobile phone B displays the screen of the recording interface 501 of computer A instead of the portrait of user A collected by mobile phone A, and the floating window 505 displays the image of user B collected by the camera of mobile phone B itself.

[0222] In some embodiments, for the convenience of user viewing, when the user clicks on the floating window 503 area on the interface of mobile phone A, the main window 502 and the floating window 503 can swap the displayed images.

[0223] like Figure 6 As shown, the computer A interface 601 displays conference materials, and the computer A can send the media stream (including audio stream, video stream, etc.) of the recording interface 601 to the mobile phone A. The mobile phone A can send the recorded media stream received from the computer A to the mobile phone B through the video call application.

[0224] In the main window 602 of the user interface of mobile phone A, the screen of the recording interface 601 of computer A is displayed, and in the floating window 603, the image of user B collected by mobile phone B through the video call application is displayed.

[0225] Mobile phone B receives the media stream from mobile phone A through the video call application connection. The main window 604 of the user interface of mobile phone B displays the screen in the recording interface 601 of computer A, and the floating window 605 displays the image of user B collected by the camera of mobile phone B. That is, the main window 602 of mobile phone A and the main window 604 of mobile phone B display the conference materials, and the main window display area is larger, which is convenient for users to watch.

[0226] In some embodiments, mobile phone A can not only send the received recorded screen of computer A to mobile phone B, but also send the images captured by the camera of mobile phone A to mobile phone B at the same time. In this way, users of mobile phone A and mobile phone B can not only see the meeting materials of computer A, but also see the avatars of the other end users, which is more convenient for visual communication between users A and B.

[0227] like Fig. 7A As shown, the computer A's interface 701 displays conference materials, and the computer A can send the media stream (including audio stream, video stream, etc.) of the recording interface 701 to the mobile phone A. The mobile phone A can send the recorded media stream received from the computer A to the mobile phone B through the video call application, and the mobile phone A can also send the image of the user A captured by its own camera to the mobile phone B.

[0228] In the main window 702 of the user interface of mobile phone A, the screen of the recording interface 701 of computer A is received. The user interface of mobile phone A may include two floating windows. In the floating window 703, the image of user B collected by mobile phone B through the video call application is displayed, and in the floating window 704, the image of user A collected by its own camera is displayed.

[0229] Mobile phone B receives the media stream from mobile phone A through the video call application connection. The main window 705 of the user interface of mobile phone B displays the screen in the recording interface 701 of computer A. The user interface of mobile phone B may include two floating windows. In the floating window 706, the image of user A captured by mobile phone A through the video call application is displayed. In the floating window 707, the image of user B captured by the camera of mobile phone B itself is displayed.

[0230] In other embodiments, mobile phone A or mobile phone B may hide the floating window where the image captured by its own camera is located to save screen display space and free up more screen area to display the main window interface.

[0231] like Figure 7B As shown, compared to Fig. 7A Only a floating window 703 is displayed in the user interface of mobile phone A. In the floating window 703, an image of user B captured by mobile phone B and received through the video call application is displayed, while the floating window 704 displaying the image captured by its own camera is hidden.

[0232] Only floating window 706 is displayed in the user interface of mobile phone B. In floating window 706, the image of user A captured by mobile phone A and received through the video call application is displayed, while floating window 707 displaying the image captured by its own camera is hidden.

[0233] In some embodiments, the user can also add editing operations on the display interface of computer A or mobile phone A, such as painting, highlighting, annotating, etc. These editing operations can be synthesized with the original video screen to generate a new media stream, which is then transmitted to mobile phone B for display.

[0234] like Fig. 8A As shown, in some embodiments, mobile phone A can add editing operations to the display screen.

[0235] refer to Fig. 8A , the interface 801 of computer A displays the conference materials, and computer A can send the media stream (including audio stream, video stream, etc.) of the recording interface 801 to mobile phone A. Mobile phone A displays the received recording interface 801 of computer A in the main window 802. The user can also touch the display screen of mobile phone A with a finger to make marks on the screen of mobile phone A, such as check mark 803. Mobile phone A can synthesize the check mark 803 on the screen with the original recording interface media stream into a new media stream, and then send the media stream with check mark 803 to mobile phone B through the video call application. The main window 804 of mobile phone B can display the received media stream of mobile phone A, which is consistent with the display content in the main window 802 of mobile phone A, that is, the content displayed includes not only the conference materials, but also the check mark 805.

[0236] like Figure 8B As shown, in other embodiments, computer A can also add editing operations to the display screen.

[0237] refer to Figure 8B, the interface 810 of computer A displays the conference materials, and the user can operate input devices such as a mouse, keyboard or stylus to make marks on the interface 810, such as a check mark 811. The interface recorded by computer A has a check mark 811. Then computer A sends the recorded media stream (including audio stream, video stream, etc.) to mobile phone A. Mobile phone A displays the received recording interface 810 of computer A in the main window 812. It can be seen that the content displayed in the main window 812 of mobile phone A also includes a check mark 813. Then mobile phone A sends the media stream with the check mark 813 to mobile phone B through the video call application. The main window 814 of mobile phone B can display the received media stream of mobile phone A, which is consistent with the display content in the main window 812 of mobile phone A, that is, the content displayed includes not only the conference materials, but also the check mark 815.

[0238] In other embodiments, computer A and mobile phone A can edit the display interface at the same time, synthesize a new media stream, and transmit it to mobile phone B for display. In this way, computer A and mobile phone A can collaboratively edit the display screen, as close as possible to the display effect that the user wants to obtain, improve the human-computer interaction function of computer A and mobile phone A, add convenience to the users of computer A and mobile phone A, and avoid the situation where the display effect cannot meet the satisfaction of both parties when the display screen is edited by computer A or mobile phone A alone.

[0239] In other embodiments, mobile phone B can also edit the display interface and synchronize it to the display interface of mobile phone A. This can also improve the human-computer interaction function of mobile phone B and add convenience to the user of mobile phone B.

[0240] It should be noted that Figure 4 , Figure 5 , Figure 6 , Fig. 7A , Figure 7B , Fig. 8A , Figure 8B The business scenarios and user interfaces shown in the document are only used to assist in describing the technical solutions of the embodiments of the present application. In actual business scenarios, the types of terminal devices included are not only mobile phones or computers, but also smart screens, electronic conference whiteboards, laptops, etc. Participants in video conferences may also include more terminal devices, and the displayed user interfaces may also be different. Figure 4 , Figure 5 , Figure 6 , Fig. 7A , Figure 7B , Fig. 8A , Figure 8B The examples shown do not constitute any limitation to the present application.

[0241] pass Figure 4 , Figure 5 , Figure 6, Fig. 7A , Figure 7B , Fig. 8A , Figure 8B In the video conferencing business scenario shown, computer A can quickly share screen interface content (such as meeting materials, photos, and videos) with mobile phones A and B for viewing. The steps are simple, fast, and efficient. It integrates the software and hardware capabilities of different devices, further improves office efficiency, and realizes a smart office scene experience.

[0242] In combination with the above embodiments, the following describes Fig.9A , Fig. 9B , Fig. 10A , Fig. 10B Some user interface schematic diagrams in the illustrated embodiments.

[0243] Mobile phone A and computer A can use short-range communication technologies such as Bluetooth, Wi-Fi, NFC, etc. to match devices and establish connections, or they can use long-range communication technologies to establish connections, and this application does not impose any restrictions on this.

[0244] exist Fig.9A , Fig. 9B , Fig. 10A , Fig. 10B In the example shown in FIG, mobile phone A and computer A establish a communication connection via Wi-Fi P2P.

[0245] If phone A and computer A are logged into the same system account or have been matched, then the other party will be identified as a trusted device.

[0246] If the other party is identified as a trusted device, the connection can be automatically allowed without the need for a tedious authentication process.

[0247] If the other device is identified as an untrusted device, authentication is required to ensure the security of the device. An untrusted device may refer to a device that has not established a communication connection, or a device that logs in to a different system account from the device. The authentication method may be scanning a QR code or entering a verification code.

[0248] In some embodiments, users can control the screen sharing process through the "Huawei Multi-Screen Collaboration" application on mobile phone A and computer A.

[0249] After mobile phone A and computer A are successfully connected, Fig.9A As shown, after detecting a user operation such as sliding down from the top status bar, mobile phone A displays a user interface 901 in response to the user operation. The user interface 901 may include: a top status bar and a notification bar 902.

[0250] Among them, the top status bar may include a mobile signal indicator, a wireless network signal indicator, a battery indicator, a time indicator, etc. The notification bar 902 may display notification items and quick setting items of "Huawei Multi-Screen Collaboration", such as the notification bar 902 showing that mobile phone A is currently connected to computer A. Of course, the name of the connected device can be customized.

[0251] In the notification bar 902, a "Start sharing screen" option 903 may also be displayed. When the user clicks the "Start sharing screen" option 903, computer A starts recording the screen and sends the recorded audio and video stream to mobile phone A, and the local audio and video channel of mobile phone A is switched to access the audio and video stream sent by computer A.

[0252] like Fig.9A As shown, in the user interface 904 of computer A, an interface 905 of the "Huawei Multi-Screen Collaboration" application is displayed. In interface 905, it is shown that the current computer A is connected to mobile phone A, and some function options are also displayed, such as "Turn on screen sharing" option 906, "Select shared file" option 907, and "Add comment box" option 908.

[0253] When the user clicks the “Start screen sharing” option 906 , computer A starts recording the screen and sends the recorded video stream to mobile phone A.

[0254] The user can also share a specified file. For example, if the user clicks the "Select Shared File" option 907 and then selects file A, then computer A only records the interface corresponding to file A, and other application interfaces will not enter the recording interface. The "Add Comment Box" function allows users to annotate content at any time when sharing the screen. The recorded interface includes the annotation content, and then the recorded audio and video stream is sent to mobile phone A, which can display the recording interface with the annotation content.

[0255] When a user selects a shared file on mobile phone A or computer A, such as a "conference PPT" file, he can Fig. 9B As shown, the user interface 910 of computer A shows that the file "Conference PPT" selected for sharing is being recorded. The user interface 911 of mobile phone A shows a notification bar 912, and the notification bar 912 shows an indication 913 of "displaying and controlling the screen of computer A". The notification bar 912 may also include a "pause screen sharing" option 914, which is convenient for the user to operate the instruction to pause screen sharing.

[0256] After the user closes the notification bar 912 by sliding up, the display interface and scene can be referred to Figure 5 As shown, mobile phone A sends the media stream (including audio stream, video stream, etc.) of the recorded screen received from computer A to mobile phone B, and the main window 504 of mobile phone B displays the video screen of the recorded screen of computer A.

[0257] In some embodiments, some options may be provided on mobile phone A to facilitate the user to control computer A on mobile phone A and provide more diverse functions for screen sharing. Fig. 10A , Fig. 10B .

[0258] like Fig. 10A As shown, computer A and mobile phone A have established a connection and are sharing the screen. Computer A records the displayed interface 1001 and sends the recorded media stream to mobile phone A. The recording interface of computer A is displayed in the main window 1002 of the display interface of mobile phone A. The display interface of mobile phone A may also include a floating window 1003 and a floating window 1004. The floating window 1003 may be used to display the image transmitted from the other end mobile phone B (not shown) of the video call connection, and the floating window 1004 may be used to display the user image captured by the camera of mobile phone A itself.

[0259] The display interface of mobile phone A may also include an option control 1005. When it is detected that the user clicks on the option control 1005, the display interface of mobile phone A may expand an option bar 1006, such as Fig. 10B As shown, the option bar 1006 may include: selecting shared files, adding comment boxes, ending screen sharing, more functional options, help and feedback, etc. The user can select options to achieve more diverse control functions.

[0260] It should be noted that Fig.9A , Fig. 9B , Fig. 10A , Fig. 10B The business scenarios and user interfaces shown in the figure are only examples, which are used to assist in describing the technical solutions of the embodiments of the present application and do not constitute any limitation to the present application. In actual business scenarios, the displayed user interfaces may be different.

[0261] Combined with the above Figure 2-10B The embodiment shown is described below. A method for sharing a screen provided by an embodiment of the present application is described below.

[0262] This embodiment is described by taking the communication system consisting of the first terminal, the second terminal, and the third terminal as an example. The embodiment of the present application does not impose any restrictions on the types of each terminal, which may be a mobile phone, a PC, a smart screen, etc. In some examples, the first terminal may be the aforementioned mobile phone A, the second terminal may be the aforementioned computer A, and the third terminal may be the aforementioned mobile phone B. It is understandable that the method provided in this embodiment is only an example and does not constitute any limitation on other embodiments of the present application. More or fewer terminal devices may be included in actual business scenarios.

[0263] Embodiment 1

[0264] Fig.11 This is a flow chart of a method for sharing a screen provided in Example 1 of the present application. In Example 1, a first terminal first establishes a second connection with a third terminal, and then the first terminal establishes a first connection with the second terminal. The specific steps are as follows:

[0265] S101: A first terminal establishes a second connection with a third terminal.

[0266] In some embodiments, the second connection established between the first terminal and the third terminal may be a video call connection or a video conference connection, which is not limited in the embodiments of the present application. The second connection may be a communication connection based on a video call application platform such as a chat APP (such as Huawei Changlian Call, WeChat, DingTalk, Enterprise WeChat, etc.), a conference APP (such as ZOOM, Welink, Cloudlink, etc.) that transmits data through the Internet or a local area network.

[0267] The first terminal or the third terminal may initiate and establish the second connection to the other terminal in response to a received user operation, such as a user operation of initiating a video call connection, a user operation of receiving a video call invitation, and the like.

[0268] S102: The first terminal plays a third media stream received from a third terminal, where the third media stream includes a picture captured by a camera of the third terminal.

[0269] S103: The third terminal plays a fourth media stream received from the first terminal, where the fourth media stream includes a picture captured by a camera of the first terminal.

[0270] After the first terminal and the third terminal establish the second connection, data streams can be sent to each other. A data stream refers to a set of data sequences that arrive in order, in large quantities, quickly, and continuously. In general, a data stream can be regarded as a dynamic data set that grows infinitely over time.

[0271] The data stream in this embodiment is a media stream, which includes an audio stream, a video stream, etc. A video stream refers to a plurality of video data, including an image stream composed of frames of images. Generally, an audio stream is also transmitted simultaneously with the video stream, that is, the audio data corresponding to the video screen.

[0272] In this embodiment, the first terminal sends a fourth media stream to the third terminal, and the fourth media stream is a data stream generated by encoding multiple images and corresponding sounds collected by the camera and microphone of the first terminal. After receiving the fourth media stream, after decoding, the third terminal plays the audio and video corresponding to the fourth media stream received from the first terminal, displays the corresponding video image, and plays the corresponding audio.

[0273] At the same time, the third terminal sends a third media stream to the first terminal, and the third media stream is a data stream generated by encoding multiple images and corresponding sounds collected by the camera and microphone of the third terminal. After receiving the third media stream, after decoding, the first terminal plays the audio and video corresponding to the third media stream received from the third terminal, displays the corresponding video image and plays the corresponding audio.

[0274] In some embodiments, the display interface of the first terminal or the third terminal includes multiple windows.

[0275] like Figure 4 As shown, the video call display interface of the first terminal or the third terminal may include a main window and a floating window, wherein the main window can be used to display the video screen sent from the opposite device, and the floating window can be used to display the video screen captured by the local camera. For example, the first user interface of the first terminal may include a main window and a floating window, wherein the main window displays the video screen corresponding to the third media stream sent by the third terminal, and the floating window displays the video screen corresponding to the fourth media stream captured by the camera of the first terminal. Similarly, the second user interface of the third terminal may also include a main window and a floating window, wherein the main window displays the video screen corresponding to the fourth media stream sent by the first terminal, and the floating window displays the video screen corresponding to the third media stream captured by the camera of the third terminal.

[0276] In some embodiments, the first user interface of the first terminal includes a first window and a second window, the first window is a main window, and the second window is a floating window. The second user interface of the third terminal includes a third window and a fourth window, the third window is a main window, and the fourth window is a floating window.

[0277] In one example, reference Figure 4 As shown, after the first terminal establishes a second connection with the third terminal, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the picture captured by the camera of the first terminal, the picture displayed in the third window is the picture captured by the camera of the first terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0278] S104: The first terminal and the second terminal establish a first connection and negotiate parameters for sharing the screen.

[0279] In an embodiment of the present application, the first connection established between the first terminal and the second terminal may be a wired connection or a wireless connection. In one implementation, the first terminal may establish a communication connection with the second terminal based on Wi-Fi near-field networking communication, such as a Wi-Fi P2P connection.

[0280] In some embodiments, the first terminal may first discover the second terminal and then send a request to the second terminal to establish a second connection. After receiving feedback information from the second terminal selected by the user indicating that the communication connection is allowed to be established, the first terminal successfully establishes the second connection with the second terminal.

[0281] Of course, the sender of the request to establish the second connection may also be the second terminal. After receiving the invitation to establish the connection sent by the second terminal, the first terminal sends a feedback to the second terminal to allow the communication connection to be established, and then the first terminal successfully establishes the second connection with the second terminal. The examples here do not limit other embodiments.

[0282] When the first terminal establishes a communication connection with the second terminal, it includes but is not limited to the following two situations:

[0283] (1) The second terminal is a trusted device

[0284] When the second terminal is a trusted device for the first terminal, after receiving the request for establishing a communication connection sent by the first terminal, the second terminal can directly send feedback information allowing the establishment of the communication connection to the first terminal to complete the establishment of the communication connection. In this way, the user's operation can be simplified and the user's control over the second terminal or the first terminal can be facilitated.

[0285] (2) The second terminal is an unauthorized device

[0286] When the second terminal is an untrusted device, when the first terminal sends a request to establish a communication connection to the second terminal, the first terminal and the second terminal enter an authentication process, and the first terminal verifies the account information of the second terminal. When the first terminal and the second terminal complete the authentication process, the communication connection is established. In this way, the security of communication between devices can be guaranteed.

[0287] In some embodiments, the authentication between the first terminal and the second terminal may be performed by scanning a code (such as a QR code, a barcode, etc.).

[0288] In other embodiments, the authentication between the first terminal and the second terminal may also be a PIN code verification. Specifically, when the first terminal initiates a request to establish a communication connection to the second terminal, the second terminal displays a PIN code for authentication in its user interface, and the first terminal may provide an input interface for inputting the PIN code. When the user enters the PIN code displayed by the second terminal on the first terminal, the authentication process between the first terminal and the second terminal is completed.

[0289] It can be seen that setting up a trusted device can optimize the user's operation of establishing a communication connection. For a device that has not established a communication connection with the first terminal, it is necessary to establish a communication connection through authentication. When the user selects a device that has established a communication connection again, it is not necessary to authenticate again. Or, when the user selects another electronic device with the same account as the first terminal to connect to the first terminal, the authentication process can be omitted. In this way, the operation is simplified and the communication security of the device is guaranteed.

[0290] It is understandable that in other embodiments of the present application, there is no need to distinguish between trusted devices and untrusted devices. All devices discovered by the first terminal can be processed according to the connection method of trusted devices or the connection method of untrusted devices.

[0291] Not limited to the above manner, the first terminal and the second terminal may also establish a communication connection in other ways, such as verification by inputting a verification code, a "touch and pay" operation based on NFC technology, etc. This embodiment of the application does not limit this.

[0292] After the first terminal and the second terminal establish a first connection, they can negotiate parameters for sharing the screen, including but not limited to the bit rate of the screen recording and audio recording modules, the recording format, the cache path, the bit rate, the frame rate, the resolution of the screen recording, the source of the recording (the audio corresponding to the media file or the audio collected by the microphone), the audio and video encoding and decoding method, the area of ​​the screen to be recorded or the designated file to be recorded, the video window rotation strategy (whether the direction of the recorded video is vertical or horizontal), the size of the recording video window, and the strategy for filling the area that does not cover the entire screen with black borders, etc. This embodiment does not impose any restrictions on this.

[0293] For example, the resolution of the video stream recorded and transmitted by the second terminal can be negotiated. Affected by the transmission channel rate, in order to ensure smooth interface display when sharing the screen, the first terminal and the second terminal can negotiate the resolution of the recorded video stream according to the current connection transmission speed. Or because the mobile phone screen is small and does not require a particularly high resolution, when the computer is used as the second terminal to record the video stream, on the premise of ensuring smooth display and a certain clarity, the resolution of the recorded video stream can be appropriately reduced to improve sharing efficiency and save bandwidth.

[0294] After the first terminal and the second terminal have established the first connection, the first terminal or the second terminal can initiate an instruction to start recording the screen. Step S105A is the first terminal initiating an instruction to start recording the screen, and step S105B is the second terminal initiating an instruction to start recording the screen, so the instruction to start recording the screen can be initiated in either step S105A or step S105B.

[0295] S105A, in response to the first user operation, the first terminal sends an instruction to start screen recording to the second terminal.

[0296] In one example, if Fig.9A As shown, when the user clicks on the "Start Sharing Screen" option 903 in the interface of mobile phone A, in response to the first user operation of clicking on the "Start Sharing Screen" option 903, the first terminal sends an instruction to start recording the screen to the second terminal.

[0297] S105B: In response to the second user operation, the second terminal detects an instruction to start recording the screen.

[0298] In one example, if Fig.9A As shown, the user can operate the mouse pointer to click on the “start sharing screen” option 906 . In response to the second user operation of clicking on the “start sharing screen” option 906 , the second terminal detects an instruction to start recording the screen.

[0299] S106: The second terminal records the screen image to generate a first media stream.

[0300] After negotiating the parameters for sharing the screen, the second terminal may respond to the instruction to start recording the screen in step S105A or step S105B to start recording the screen and generate a first media stream.

[0301] The first media stream may include a display image on the screen of the second terminal, and audio collected by a microphone of the second terminal or sound corresponding to a media file played by the second terminal.

[0302] In some embodiments, after the first terminal and the second terminal establish a first connection, screen recording can be started automatically without the user issuing an instruction to start screen recording.

[0303] The screen recorded by the second terminal may be a full-screen interface of the second terminal, or the user may specify the content to be recorded in order to protect privacy, such as a screen area specified by the user, or one or more documents, pictures, etc. specified by the user.

[0304] In some embodiments, the second terminal can also turn on the local camera, synthesize the picture captured by the local camera with the picture of the recorded screen, generate a first media stream, and send it to the first terminal together. In this way, not only can you see the content of the recorded screen, but you can also see the picture captured by the camera of the second terminal, such as the image of the user using the second terminal.

[0305] In some embodiments, the second terminal may also synthesize the real-time annotations of the user on the second terminal with the picture of the recorded screen to generate a first media stream, and send it to the first terminal together, thereby improving the efficiency of screen sharing.

[0306] S107: The second terminal sends a first media stream to the first terminal.

[0307] The second terminal sends a first media stream to the first terminal through the first connection.

[0308] S108: The first terminal generates a second media stream according to the acquired first media stream.

[0309] In some embodiments, after the first terminal obtains the first media stream (including audio data stream and video data stream) from the second terminal, it forwards the first media stream to the third terminal, that is, the second media stream is equal to the first media stream.

[0310] In some embodiments, after the first terminal obtains the first media stream from the second terminal, it can perform new editing on the basis of the first media stream to generate a second media stream and send it to the third terminal.

[0311] For example, the first terminal can package the first media stream of the screen recorded by the second terminal and the user image captured by the camera of the first terminal to generate a second media stream and send it to the third terminal. Fig. 7A As shown, mobile phone B can not only see the conference material screen shared by computer A, but also see the user image captured by the camera of mobile phone A.

[0312] For another example, the first terminal can write or annotate on the recorded screen corresponding to the first media stream, and then generate a second media stream, in which the video screen in the second media stream has editing traces such as writing or annotating. The first terminal sends the second media stream to the third terminal, and the video screen corresponding to the second media stream displayed by the third terminal shows the screen with editing traces. Fig. 8A As shown, mobile phone B can not only see the conference material screen shared by computer A, but also see the newly generated graffiti and annotation traces by mobile phone A.

[0313] In some embodiments, the audio data of the second media stream may be selected from audio data transmitted by the second terminal, such as sound collected by a microphone of the second terminal, or audio corresponding to a media file played by the second terminal.

[0314] In other embodiments, the audio data of the second media stream may be selected to come from the first terminal, such as the user's voice collected by a microphone of the first terminal, or the audio corresponding to a media file on the first terminal.

[0315] S109: The first terminal sends a second media stream to the third terminal.

[0316] Through the video call APP, the first terminal sends the second media stream to the third terminal.

[0317] Specifically, the first terminal can use the local camera and microphone to transmit the audio and video transmission channel of the second media stream, and then transmit the second media stream to the upper-layer video call software. For the upper-layer video call application, it is unaware that the video stream has changed, and there is no need to modify the system interface called by the third-party application, that is, there is no need for the third-party application to make adaptation modifications, and the first terminal can send the screen image captured by the second terminal to the third terminal.

[0318] S110: The first terminal plays a third media stream received from a third terminal, where the third media stream includes a picture captured by a camera of the third terminal.

[0319] S111, the third terminal plays a second media stream received from the first terminal, where the second media stream includes a screen image captured by a screen recording of the second terminal.

[0320] After the above steps, the picture displayed by the third terminal includes the screen picture collected by the screen recording of the second terminal.

[0321] In some embodiments, the display interface of the first terminal or the third terminal includes multiple windows.

[0322] like Figure 5 As shown, the video call display interface of the first terminal or the third terminal may include a main window and a floating window, wherein the main window can be used to display the video screen sent from the opposite device, and the floating window can be used to display the video screen captured by the local camera. For example, the second user interface of the first terminal may include a main window and a floating window, wherein the main window displays the video screen corresponding to the third media stream sent by the third terminal, and the floating window displays the screen from the recording of the second terminal. Similarly, the fourth user interface of the third terminal may also include a main window and a floating window, wherein the main window displays the video screen corresponding to the second media stream sent by the first terminal, including the screen from the recording of the second terminal, and the floating window displays the video screen corresponding to the third media stream captured by the local camera of the third terminal.

[0323] In other embodiments, the third terminal not only displays the screen recording from the second terminal, but also displays the user image captured by the camera of the first terminal. Fig. 7A As shown, in the user interface of mobile phone B, not only the screen of the conference material recorded by computer A is displayed, but also the image of user A of mobile phone A is displayed in the floating window 706.

[0324] In other embodiments, the third terminal not only displays the screen recording of the second terminal, but also displays the editing traces such as graffiti and annotations made by the first terminal. As shown in 8A, the user interface of mobile phone B not only displays the screen recording of the conference data of computer A, but also synchronously displays the check mark 805 made by the user on the mobile phone A side.

[0325] In one example, reference Figure 5 As shown, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0326] In one example, reference Figure 6 As shown, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0327] In one example, reference Fig. 7A As shown, the first user interface of the first terminal also includes a fifth window, and the second user interface of the third terminal also includes a sixth window, and the fifth window and the sixth window are floating windows. The picture displayed in the fifth window is the picture collected by the camera of the first terminal, and the picture displayed in the sixth window is the picture collected by the camera of the first terminal.

[0328] Embodiment 2

[0329] Fig.12 This is a flow chart of the method for sharing a screen provided in Example 2 of the present application. In Example 2, the first terminal first establishes a first connection with the second terminal, and then the first terminal establishes a second connection with the third terminal. That is to say, the first terminal first establishes a shared screen connection with the second terminal, and then establishes a video call connection with the third terminal. In this way, on the interface of the third terminal, the screen captured from the recorded screen of the second terminal can be directly received and displayed from the first terminal. The camera of the first terminal does not need to be turned on, which further improves the efficiency of sharing the screen.

[0330] The specific steps are as follows:

[0331] S201: A first terminal and a second terminal establish a first connection and negotiate parameters for sharing a screen.

[0332] In an embodiment of the present application, the first connection established between the first terminal and the second terminal may be a wired connection or a wireless connection. In one implementation, the first terminal may establish a communication connection with the second terminal based on Wi-Fi near-field networking communication, such as a Wi-Fi P2P connection.

[0333] For more details, please refer to step S104, which will not be elaborated here.

[0334] After the first terminal and the second terminal establish a first connection, they can negotiate parameters for sharing the screen, including but not limited to the bit rate of the screen recording and audio recording modules, the recording format, the cache path, the bit rate, the frame rate, the resolution of the screen recording, the source of the recording (the audio corresponding to the media file or the audio collected by the microphone), the audio and video encoding and decoding method, the area of ​​the screen to be recorded or the designated file to be recorded, the video window rotation strategy (whether the direction of the recorded video is vertical or horizontal), the size of the recording video window, and the strategy for filling the area that does not cover the entire screen with black borders, etc. This embodiment does not impose any restrictions on this.

[0335] For example, the resolution of the video stream recorded and transmitted by the second terminal can be negotiated. Affected by the transmission channel rate, in order to ensure smooth interface display when sharing the screen, the first terminal and the second terminal can negotiate the resolution of the recorded video stream according to the current connection transmission speed. Or because the mobile phone screen is small and does not require a particularly high resolution, when the computer is used as the second terminal to record the video stream, on the premise of ensuring smooth display and a certain clarity, the resolution of the recorded video stream can be appropriately reduced to improve sharing efficiency and save bandwidth.

[0336] After the first terminal and the second terminal have established a first connection, the first terminal or the second terminal can initiate an instruction to start recording the screen. Step S202A is the first terminal initiating an instruction to start recording the screen, and step S202B is the second terminal initiating an instruction to start recording the screen, so the instruction to start recording the screen can be initiated in either step S202A or step S202B.

[0337] S202A, in response to a first user operation, the first terminal sends an instruction to start screen recording to the second terminal.

[0338] In one example, if Fig.9A As shown, when the user clicks on the "Start Sharing Screen" option 903 in the interface of mobile phone A, in response to the first user operation of clicking on the "Start Sharing Screen" option 903, the first terminal sends an instruction to start recording the screen to the second terminal.

[0339] S202B: In response to the second user operation, the second terminal detects an instruction to start recording the screen.

[0340] In one example, if Fig.9A As shown, the user can operate the mouse pointer to click on the “start sharing screen” option 906 . In response to the second user operation of clicking on the “start sharing screen” option 906 , the second terminal detects an instruction to start recording the screen.

[0341] S203: The second terminal records the screen image to generate a first media stream.

[0342] After negotiating the parameters for sharing the screen, the second terminal may respond to the instruction to start recording the screen in step S202A or step S202B to start recording the screen and generate a first media stream.

[0343] The first media stream may include a display image on the screen of the second terminal, and audio collected by a microphone of the second terminal or sound corresponding to a media file played by the second terminal.

[0344] In some embodiments, after the first terminal and the second terminal establish a first connection, screen recording can be started automatically without the user issuing an instruction to start screen recording.

[0345] The screen recorded by the second terminal may be a full-screen interface of the second terminal, or the user may specify the content to be recorded in order to protect privacy, such as a screen area specified by the user, or one or more documents, pictures, etc. specified by the user.

[0346] In some embodiments, the second terminal can also turn on the local camera, synthesize the picture captured by the local camera with the picture of the recorded screen, generate a first media stream, and send it to the first terminal together. In this way, not only can you see the content of the recorded screen, but you can also see the picture captured by the camera of the second terminal, such as the image of the user using the second terminal.

[0347] In some embodiments, the second terminal may also synthesize the real-time annotations of the user on the second terminal with the picture of the recorded screen to generate a first media stream, and send it to the first terminal together, thereby improving the efficiency of screen sharing.

[0348] S204: The second terminal sends a first media stream to the first terminal.

[0349] The second terminal sends a first media stream to the first terminal through the first connection.

[0350] S205: The first terminal establishes a second connection with the third terminal.

[0351] In some embodiments, the second connection established between the first terminal and the third terminal may be a video call connection or a video conference connection, which is not limited in the embodiments of the present application. The second connection may be a communication connection based on a video call application platform such as a chat APP (such as Huawei Changlian Call, WeChat, DingTalk, Enterprise WeChat, etc.), a conference APP (such as ZOOM, Welink, Cloudlink, etc.) that transmits data through the Internet or a local area network.

[0352] The first terminal or the third terminal may initiate and establish the second connection to the other terminal in response to a received user operation, such as a user operation of initiating a video call connection, a user operation of receiving a video call invitation, and the like.

[0353] S206: The first terminal generates a second media stream according to the acquired first media stream.

[0354] In some embodiments, after the first terminal obtains the first media stream from the second terminal, it forwards the first media stream to the third terminal, that is, the second media stream is equal to the first media stream.

[0355] In some embodiments, after the first terminal obtains the first media stream from the second terminal, it can perform new editing on the basis of the first media stream to generate a second media stream and send it to the third terminal.

[0356] For example, the first terminal can package the first media stream of the screen recorded by the second terminal and the user image captured by the camera of the first terminal to generate a second media stream and send it to the third terminal. Fig. 7A As shown, mobile phone B can not only see the conference material screen shared by computer A, but also see the user image captured by the camera of mobile phone A.

[0357] For another example, the first terminal can write or annotate on the recorded screen corresponding to the first media stream, and then generate a second media stream, in which the video screen in the second media stream has editing traces such as writing or annotating. The first terminal sends the second media stream to the third terminal, and the video screen corresponding to the second media stream displayed by the third terminal shows the screen with editing traces. Fig. 8A As shown, mobile phone B can not only see the conference material screen shared by computer A, but also see the newly generated graffiti and annotation traces by mobile phone A.

[0358] S207: The first terminal sends a second media stream to the third terminal.

[0359] Through the video call APP, the first terminal sends the second media stream to the third terminal.

[0360] Specifically, the first terminal can use the local camera and microphone to transmit the audio and video transmission channel of the second media stream, and then transmit the second media stream to the upper-layer video call software. For the upper-layer video call application, it is unaware that the video stream has changed, and there is no need to modify the system interface called by the third-party application, that is, there is no need for the third-party application to make adaptation modifications, and the first terminal can send the screen image captured by the second terminal to the third terminal.

[0361] S208: The first terminal plays a third media stream received from the third terminal, where the third media stream includes a picture captured by a camera of the third terminal.

[0362] S209: The third terminal plays a second media stream received from the first terminal, where the second media stream includes a screen image captured by a screen recording of the second terminal.

[0363] After the above steps, the picture displayed by the third terminal includes the screen picture collected by the screen recording of the second terminal.

[0364] In some embodiments, the display interface of the first terminal or the third terminal includes multiple windows.

[0365] like Figure 5 As shown, the video call display interface of the first terminal or the third terminal may include a main window and a floating window, wherein the main window can be used to display the video screen sent from the opposite device, and the floating window can be used to display the video screen captured by the local camera. For example, the second user interface of the first terminal may include a main window and a floating window, wherein the main window displays the video screen corresponding to the third media stream sent by the third terminal, and the floating window displays the screen from the recording of the second terminal. Similarly, the fourth user interface of the third terminal may also include a main window and a floating window, wherein the main window displays the video screen corresponding to the second media stream sent by the first terminal, including the screen from the recording of the second terminal, and the floating window displays the video screen corresponding to the third media stream captured by the local camera of the third terminal.

[0366] In other embodiments, the third terminal not only displays the screen recording from the second terminal, but also displays the user image captured by the camera of the first terminal. Fig. 7A As shown, in the user interface of mobile phone B, not only the screen of the conference material recorded by computer A is displayed, but also the image of user A of mobile phone A is displayed in the floating window 706.

[0367] In other embodiments, the third terminal not only displays the screen recording of the second terminal, but also displays the editing traces such as graffiti and annotations made by the first terminal. As shown in 8A, the user interface of mobile phone B not only displays the screen recording of the conference data of computer A, but also synchronously displays the check mark 805 made by the user on the mobile phone A side.

[0368] In some embodiments, the first user interface of the first terminal includes a first window and a second window, the first window is a main window, and the second window is a floating window. The second user interface of the third terminal includes a third window and a fourth window, the third window is a main window, and the fourth window is a floating window.

[0369] In one example, reference Figure 5 As shown, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the picture captured by the camera of the third terminal, the picture displayed in the second window is the screen picture recorded by the second terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0370] In one example, reference Figure 6 As shown, after the first terminal sends the second media stream to the third terminal through the second connection, the picture displayed in the first window is the screen picture recorded by the second terminal, the picture displayed in the second window is the picture captured by the camera of the third terminal, the picture displayed in the third window is the screen picture recorded by the second terminal, and the picture displayed in the fourth window is the picture captured by the camera of the third terminal.

[0371] In one example, reference Fig. 7A As shown, the first user interface of the first terminal also includes a fifth window, and the second user interface of the third terminal also includes a sixth window, and the fifth window and the sixth window are floating windows. The picture displayed in the fifth window is the picture collected by the camera of the first terminal, and the picture displayed in the sixth window is the picture collected by the camera of the first terminal.

[0372] By implementing the method provided in this embodiment, the second terminal can quickly share the screen interface content (such as meeting materials, photos, videos) with the first terminal and the third terminal for viewing without installing video call software. The steps are simple, fast and efficient, further improving communication efficiency and user communication experience.

[0373] In combination with the foregoing embodiment, the virtual device architecture of the communication system 80 provided in the embodiment of the present application is exemplarily introduced below.

[0374] Fig.1380 is a software module diagram of the communication system 80 provided in the embodiment of the present application. The communication system 80 is composed of a first terminal and a second terminal. This embodiment does not limit the device type and operating system of the first terminal and the second terminal. The software system of the first terminal or the second terminal includes but is not limited to Or other operating systems.

[0375] like Fig.13 As shown, the first terminal establishes a first connection with the second terminal. The first terminal may include a communication module 1110, a video call application 1111, a display module 1112, an audio and video switching module 1113, a policy management module 1114, a camera management module 1115, a microphone management module 1116, etc. The second terminal may include a display module 1101, a communication module 1102, a screen recording module 1103, a recording module 1104, a policy management module 1105, etc. Modules of the same terminal may communicate with each other through a software interface (application programming interface, API).

[0376] On the first terminal side, the communication module 1110 is responsible for receiving the audio and video streams sent by the second terminal.

[0377] The video call application 1111 is used to establish a video call connection with other terminal devices. As an upper-layer application, the video call application 1111 simply calls the local microphone and camera, and does not perceive that the lower-layer audio and video stream is replaced by the screen recording of the second terminal.

[0378] The display module 1112 is responsible for displaying images.

[0379] The audio and video switching module is responsible for injecting the received audio and video content into the camera channel and microphone channel of the first terminal respectively.

[0380] The policy management module 1114 is responsible for negotiating with the second terminal about parameters of the shared screen, such as audio and video bitrate, video resolution, shared screen area, video window rotation policy, and other information.

[0381] The camera management module 1115 and the microphone management module 1116 are responsible for decoding the audio and video streams sent by the second terminal and then passing them to the video call application 1111.

[0382] On the second terminal side, the display module 1101 is responsible for displaying images.

[0383] The communication module 1102 is responsible for transmitting the recorded screen content to the first terminal.

[0384] The screen recording module 1103 is responsible for recording screen content.

[0385] The recording module 1104 is responsible for recording the output audio of the screen or the input audio of the microphone.

[0386] The policy management module 1105 is responsible for coordinating and managing the bit rate of the screen recording and audio recording modules, the resolution of the screen recording, the source of the recording (screen audio or microphone audio), the audio and video encoding and decoding method, the recording screen area or the recording specified file, the video window rotation strategy and other policy parameters.

[0387] By implementing the embodiment shown in the communication system 80, the second terminal can quickly share the screen interface content with the first terminal and the third terminal, and the steps are simple, fast and efficient, which further improves office efficiency. For example, the first terminal and the second terminal can support the multi-screen collaboration function, and the second terminal shares the screen interface content with the video call application on the first terminal. When the third terminal is in a video call with the first terminal, the third terminal can watch the interface content shared by the second terminal through the video call application.

[0388] The display content of the media stream (including audio stream and video stream) sent by the first terminal to the third terminal includes but is not limited to the following three situations:

[0389] 1. The displayed content is edited by the first terminal.

[0390] After the first terminal receives the video stream collected by the second terminal, the user can perform some control operations on the first terminal to edit, set or adjust the video stream, generate a new video stream, and then send it to the third terminal.

[0391] For example, the user can make some annotations on the video screen of the first terminal, and then these annotations are combined with the original video screen to form a new video screen and sent to the third terminal. For another example, the user can set the video screen sent by the first terminal to the third terminal to include not only the screen recording screen of the second terminal, but also the user's avatar captured by the camera of the first terminal. In this way, the third terminal side not only displays the screen recording screen from the second terminal, but also displays the avatar of the user of the first terminal, so that the user's video call experience is better. In addition, the picture parameters can also be adjusted on the video call interface of the first terminal. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc.

[0392] In this way, the first terminal can adjust the image captured by the second terminal using its own editing capabilities, generate a new display screen and send it to the third terminal, which increases the convenience of use for the user of the first terminal.

[0393] 2. The displayed content is edited by the second terminal.

[0394] After the second terminal generates the screen recording, the user can perform some control operations on the second terminal to edit, set or adjust the video stream, generate a new video stream, and then send it to the first terminal.

[0395] For example, the user can make some annotations on the screen recording of the second terminal, and then these annotations together with the original screen recording are synthesized into a new video screen and sent to the first terminal, and then sent by the first terminal to the third terminal. For another example, the user can set the video screen sent by the second terminal to the first terminal to include not only the screen recording of the second terminal, but also the user's avatar captured by the camera of the second terminal. In this way, the first terminal side and the third terminal side not only display the screen recording from the second terminal, but also display the avatar of the user of the second terminal, so that the user's video call experience is better. In addition, the picture parameters can also be adjusted on the user interface of the second terminal. The picture parameters may include one or more of the following: image cropping size, image color temperature, beauty / body type, filter type, sticker options, etc.

[0396] In this way, the second terminal can use its own editing capabilities to adjust the images it has collected, generate a new display screen and send it to the first terminal, which adds convenience to the user of the second terminal.

[0397] 3. The displayed content is collaboratively edited by the first terminal and the second terminal.

[0398] In combination with the description of the first two aspects, in one implementation, both the first terminal and the second terminal can have the function of editing the video screen. In this way, the first terminal and the second terminal can collaboratively complete the editing of the video screen, as close as possible to the display effect that the user wants to obtain, which adds convenience to the users of the first terminal and the second terminal, and avoids the situation where the editing display effect cannot satisfy both parties when the first terminal or the second terminal edits the display screen separately.

[0399] For example, the first terminal can edit the edited screen again based on the editing of the screen recording screen by the second terminal. After the first terminal or the second terminal edits the screen recording screen, the display screen is synchronized on the display interface of the first terminal and the second terminal.

[0400] Furthermore, after the first terminal sends an instruction to adjust the display effect to the second terminal, the second terminal can also perform compatibility processing on the instruction so that the second terminal can respond to the processed instruction and effectively complete the adjustment of the corresponding display effect. This is because the device types and device models of the first terminal and the second terminal may not be the same. When the first terminal responds to the user's operation of adjusting the display effect, some instructions that the second terminal does not support or cannot recognize may be generated, and the second terminal may not be able to directly respond to these instructions to adjust the display effect of the interface. The second terminal's compatibility processing of the instruction enables the second terminal to respond to the user's operation on the first terminal, and realizes the first terminal's control over the second terminal's adjustment of the display interface effect.

[0401] From the above situation, it can be seen that not only the second terminal can have the ability to edit the screen recording screen or other video streams, but the first terminal can also have the ability to edit the screen recording screen or other video streams from the second terminal. This strengthens the control effect of the first terminal over the display interface of the second terminal and improves the user experience.

[0402] In an embodiment of the present application, the first terminal displays the image of the second terminal by using the data stream channel collected by its own camera to transmit the image from the second terminal, that is, the first terminal originally uses the data stream channel to transmit the data of the image collected by its own camera, but in the scenario of the embodiment of the present application, the data stream channel is used to transmit the screen recording data stream or other video stream from the second terminal. In this way, the upper-layer application of the first terminal does not need to perceive whether the displayed image comes from the first terminal or the second terminal. The video stream collected by the second terminal is equivalent to the video stream collected by a remote camera of the first terminal. There is no need to modify the way the third-party application calls the system interface, that is, there is no need for the third-party application to adapt. The first terminal can send the acquired image of the second terminal to the upper-layer application, which expands the use scenario of screen sharing between the first terminal and the second terminal.

[0403] The above introduction to the virtual device architecture of the communication system 80 is only an example. It is understood that the software architecture illustrated in the embodiment of the present invention does not constitute a specific limitation on the present application. In other embodiments of the present application, the software architecture of the communication system 80 may include more or fewer modules than shown in the figure, or combine certain modules, or split certain modules, or have different architecture arrangements. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0404] Combination Fig.13 The software module diagram of the communication system 80 is shown, and the process of an embodiment is described below from the perspective of module functions.

[0405] refer to Fig.14, Fig.14 A process step of sharing a screen is shown.

[0406] S301: A user initiates a video call in a video call application of a first terminal.

[0407] This embodiment does not limit the video call application. The video call application can be a social application with a video chat function, such as WeChat, QQ, Changlian Call, DingTalk, etc., or it can be a video conferencing application, such as ZOOM, welink, cloudlink, etc.

[0408] S302: The video call application calls the local camera of the first terminal.

[0409] S303: The video call application calls the local microphone of the first terminal.

[0410] When a user uses a video call application to make a video call, the video call application can call the local camera and microphone to collect audio and image data and send it to the other end of the video call.

[0411] S304: The user initiates a screen sharing function on the first terminal.

[0412] refer to Fig.9A In the illustrated embodiment, after the first terminal establishes a connection with the second terminal, the user can select the “Start Sharing Screen” option 903 , and then the first terminal sends an instruction to start recording the screen to the second terminal.

[0413] The screen sharing function enables the first terminal to obtain the screen of the second terminal, and then the first terminal displays the screen of the second terminal. After the first terminal obtains the screen of the second terminal, it can send it to the third terminal, and then the third terminal can display the screen of the second terminal. In some embodiments, the first terminal can control the screen of the second terminal.

[0414] S305: The audio and video switching module of the first terminal negotiates screen sharing parameters with the policy management module.

[0415] After the audio and video switching module receives the user-initiated instruction to start screen sharing, the audio and video switching module of the first terminal can negotiate with the policy management module on the parameters of screen sharing, including but not limited to the bit rate of the screen recording and audio recording modules, the resolution of the screen recording, the source of the recording (screen audio or microphone audio), the audio and video encoding and decoding method, the area of ​​the screen recorded or the designated file recorded, the video window rotation strategy (whether the recorded video is vertical or horizontal), the size of the recording video window, and filling the area that does not cover the entire screen with black borders, etc.

[0416] S306: The policy management module of the first terminal sends audio and video parameters of the shared screen to the policy management module of the second terminal.

[0417] S307, the policy management module of the second terminal sends a screen recording instruction to the screen recording module.

[0418] S308: The policy management module of the second terminal sends an audio recording instruction to the recording module.

[0419] The policy management module of the second terminal sends instructions for recording audio and video streams to the screen recording module and the audio recording module according to the negotiated audio and video parameters of the shared screen.

[0420] S309, the screen recording module of the second terminal uploads the screen recording video stream to the policy management module.

[0421] S310, the recording module of the second terminal uploads the screen recording audio stream to the policy management module.

[0422] S311, the policy management module of the second terminal encodes and merges the screen recording audio and video streams.

[0423] After receiving the screen recording video stream and the screen recording audio stream, the policy management module of the second terminal will merge and integrate to generate the screen recording audio and video stream. For example, the second terminal can merge the video screen of the recorded screen with the image captured by the camera of the second terminal into one screen and integrate it into an audio and video stream.

[0424] S312, the policy management module of the second terminal sends the screen recording audio and video stream to the communication module.

[0425] S313, the communication module of the second terminal sends the screen recording audio and video stream to the communication module of the first terminal.

[0426] S314, the communication module of the first terminal reports the screen recording audio and video stream to the audio and video switching module.

[0427] S315, the audio and video switching module of the first terminal sends an audio switching instruction to the microphone.

[0428] S316, the audio and video switching module of the first terminal sends a video switching instruction to the camera.

[0429] The audio and video switching module can inject the received audio and video content into the camera channel and microphone channel of the first terminal respectively, and replace the audio and video stream collected by the local camera and microphone of the first terminal with the audio and video stream received from the second terminal.

[0430] S317, the microphone management module of the first terminal uploads the audio stream to the video call application.

[0431] S318, the camera management module of the first terminal uploads the video stream to the video call application.

[0432] As an upper-layer application, the video call application simply calls the local microphone and camera, and is unaware that the lower-layer audio and video stream has been replaced by the screen recording audio and video stream of the second terminal.

[0433] Implementation Fig.13 , Fig.14 In the illustrated embodiment, the second terminal can quickly share the screen interface content (such as conference materials, photos, and videos) with the first terminal without installing video call application software. The first terminal can transmit the screen recording audio and video of the second terminal to other terminals for viewing. The steps are simple, fast and efficient, which further improves communication efficiency and user communication experience.

[0434] In combination with the foregoing embodiments, an embodiment of screen sharing between mobile phone A and computer A is described below. Fig.15 The process of screen sharing between mobile phone A and computer A described in this embodiment is shown.

[0435] Step 1: The video call application on mobile phone A initiates a video call and establishes a video call channel.

[0436] This embodiment does not limit the video call application. The video call application can be a social application with a video chat function, such as WeChat, QQ, Changlian Call, DingTalk, etc., or it can be a video conferencing application, such as ZOOM, welink, cloudlink, etc.

[0437] During a video call, the video call application can send the received video data to the streaming engine, which processes and displays it on the display screen. The streaming engine may include encoding function, decoding function, image rendering function, image capture function, streaming protocol, quality of service (QoS) technology, etc.

[0438] Step ②: Mobile phone A and computer A establish a multi-screen collaborative connection.

[0439] The multi-screen collaboration function refers to a distributed communication technology. The multi-screen collaboration function can realize cross-operating system and cross-device collaborative work. For example, after mobile phone A and computer A establish a multi-screen collaborative connection, mobile phone A and computer A can realize screen sharing, resource sharing, collaborative operation and other functions to improve data processing efficiency. For example, in this embodiment, after the multi-screen collaborative connection is established, computer A can share its own screen image with mobile phone A, and mobile phone A can display the screen image of computer A.

[0440] Step 3: The user selects a media stream for sharing the screen on mobile phone A.

[0441] In some embodiments, the user can send instructions on mobile phone A to start screen sharing. The user can also select the media stream of the shared screen, that is, the video stream and / or audio stream of computer A. For example, the video stream can be the image stream captured by the camera of computer A, or the video stream of the screen recorded by computer A. The audio stream can be the sound captured by the microphone of computer A, or the audio corresponding to the media file played by computer A.

[0442] After the user chooses to start sharing the screen, mobile phone A and computer A can negotiate the parameters for recording audio and video, including but not limited to the bit rate of the screen recording and recording modules, recording format, cache path, bit rate, frame rate, resolution of the screen recording, source of the recording (audio corresponding to the media file or audio collected by the microphone), audio and video encoding and decoding method, area of ​​the screen recording or recording a specified file, video window rotation strategy (whether the direction of the recorded video is vertical or horizontal), size of the recorded video window, and strategy for filling black borders for areas that do not cover the entire screen, etc. This embodiment does not impose any restrictions on this.

[0443] Step ④: The user selects the audio and video stream to be shared on computer A.

[0444] In some embodiments, the user selects a shared audio and video stream on computer A. For example, the user can choose to share the recorded video stream of the PC desktop, or can share a specified file, or can choose to synthesize the image captured by the camera of computer A and the recorded video stream of the desktop into a new audio and video stream for sharing.

[0445] Step ⑤: The codec of computer A can encode and decode the audio and video streams and compress them.

[0446] This embodiment does not limit the encoding and decoding method of the media stream.

[0447] Step ⑥: Computer A sends the audio and video stream to the screen projection transmission module.

[0448] Step 7: Computer A transmits the recorded audio and video stream to mobile phone A.

[0449] Computer A is the screen projection source, and mobile phone A is the screen projection receiver. Computer A can send its own screen data to mobile phone A, and mobile phone A can view the screen interface of computer A on the screen.

[0450] Step ⑧: After receiving the recorded audio and video from computer A, mobile phone A sends the virtual video stream to the video call application.

[0451] Specifically, mobile phone A can use the camera channel and microphone channel of mobile phone A to transmit the audio and video data received from computer A to the upper-layer video call application. As an upper-layer application, the video call application simply calls the local microphone and camera, and does not perceive that the lower-layer audio and video stream has changed from the audio and video stream collected by its own camera and microphone to the audio and video stream from the recorded screen of computer A.

[0452] When mobile phone A and mobile phone B make a video call through a video call application, mobile phone A can transmit the screen image of computer A received to mobile phone B through the video call application connection. The audio and video stream transmitted by mobile phone A to the video call application is the media stream data from the local camera channel and microphone channel. Mobile phone A occupies the local camera channel and microphone channel with the virtual audio and video stream from computer A. Therefore, the image and sound from mobile phone A seen on mobile phone B should be the image and sound collected by the camera and microphone of mobile phone A, but they are replaced by the audio and video stream from computer A.

[0453] Furthermore, mobile phone A can also edit the screen image recorded by computer A, so that the user can change the display effect of the screen image on mobile phone A, such as writing and annotating on the screen image. The user can also select the source of the audio stream transmitted to mobile phone B on mobile phone A, such as the audio stream can be the sound collected by the microphone of computer A, or the sound collected by the microphone of mobile phone A, or the audio corresponding to the media file on computer A, or the audio corresponding to the media file on mobile phone A. For details, please refer to the above embodiments, which will not be repeated here.

[0454] Implementation Fig.15 In the illustrated embodiment, computer A can quickly share screen interface content (such as meeting materials, photos, and videos) with mobile phone A without installing video call application software. Mobile phone A can use the video call application software to transmit the audio and video recorded by computer A to other terminals for viewing. The steps are simple, fast, and efficient, which further improves communication efficiency and user communication experience.

[0455] The implementation methods described in the above embodiments are only exemplary and do not constitute any limitation on other embodiments of the present application. The specific internal implementation method may vary depending on the type of electronic device, the operating system installed, the program used, and the interface called. The present application does not impose any limitation on the implementation methods, and the characteristic functions described in the embodiments of the present application can be realized.

[0456] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0457] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0458] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0459] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0460] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for sharing a screen, characterized in that: The method comprises: The first terminal establishes a second connection with the third terminal, where the second connection is a video call connection established through a video call application APP; The first terminal establishes a first connection with a second terminal, and the second terminal and the first terminal are logged in with the same account; The first terminal triggers the second terminal to share the screen interface content of the second terminal through the video call application APP in response to the first operation, so that the third terminal displays the screen interface content of the second terminal when making a video call with the first terminal; The first terminal receives an operation on the content shared during the video call from the third terminal so that the shared content remains consistent, or the first terminal sends an operation on the content shared during the video call from the second terminal to the third terminal so that the shared content remains consistent.

2. The method according to claim 1, characterized in that The method further comprises: The first terminal plays the picture collected by the camera of the third terminal and receives the picture from the third terminal, and sends the picture collected by the camera of the first terminal to the third terminal; The first terminal displays a first user interface, which includes a first window and a second window. The picture displayed in the first window is the picture captured by the camera of the third terminal, and the picture displayed in the second window is the picture captured by the camera of the first terminal.

3. The method according to claim 1, characterized in that The second connection is a communication connection for transmitting data via the Internet or a local area network.

4. The method according to claim 1, characterized in that: The first operation includes an operation performed by a user on a first option displayed by the first terminal.

5. The method according to claim 1, characterized in that The first operation includes an operation performed by a user on a second option displayed by the second terminal.

6. The method according to claim 2, characterized in that After the first terminal triggers the second terminal to share the screen interface content of the second terminal through the video call application APP in response to the first operation, the method further includes: The first terminal displays the screen interface content of the second terminal in the first window, and displays the picture collected by the camera of the third terminal in the second window.

7. The method according to claim 2, characterized in that: The first window is a main window, the second window is a floating window, and a size of the second window is smaller than a size of the first window.

8. The method according to claim 1, characterized in that The first terminal and the second terminal are mutually trusted devices; and the first terminal and the second terminal establish a first connection, specifically including: The first terminal automatically establishes the first connection with the second terminal.

9. The method according to any one of claims 1 to 8, characterized in that The screen interface content of the second terminal includes conference materials, photos or videos in the second terminal.

10. The method according to any one of claims 1 to 8, characterized in that The screen interface content of the second terminal includes a picture of the screen recorded by the second terminal.

11. An electronic device, characterized in that: The electronic device includes: a communication device, a display screen, a memory, and a processor coupled to the memory, multiple applications, and one or more programs; the display screen is used to display images, the memory stores computer executable instructions, and when the processor executes the instructions, the electronic device implements the method described in any one of claims 1-10.

12. A communication system, characterized in that: The communication system comprises: a first terminal, a second terminal and a third terminal, wherein the first terminal executes the method according to any one of claims 1 to 10.

13. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

14. A computer program product comprising instructions, characterized in that When the computer program product runs on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 10.

Images