A method for adjusting screen projection display parameters

Abstract

The present application provides a method for adjusting screen projection display parameters. The method includes: a large-screen device establishes a first screen projection connection with an electronic device, and the electronic device establishes a second screen projection connection with a display device. The large-screen device will negotiate the target display parameters based on the display parameters of the large-screen device, the display parameters of the electronic device, and the display parameters of the display device. The target display parameters include a target encoding frame rate, so that the target encoding frame rate is less than the display frame rate or decoding frame rate of the display device, or the display frame rate of the large-screen device 200 is greater than the target encoding frame rate. Afterwards, the large-screen device will process the projection content according to the target display parameters and send it to the display device. In this way, the problem that the projection content displayed on the display screen is out of sync with the audio when an external display screen is connected to the projection system is solved, and user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of screen projection technology, and in particular to a method for adjusting screen projection display parameters. Background Art

[0002] With the development of network communication technology, the large screen features of smart TVs and the portability of smartphones and tablets have helped improve the user experience through multi-screen collaboration. For example, when a mobile phone and a personal computer (PC) are coordinating screen projection, the phone and PC will negotiate the display frame rate. After that, the phone and PC will display the projected content according to the negotiated display frame rate.

[0003] Later, when the PC needs to expand its display, if the external display does not have an audio playback module, it will only display the image transmitted by the PC. If the display frame rate of the external display is lower than the display frame rate negotiated between the phone and PC, the image displayed on the external display will be inconsistent with the image displayed on the PC, causing the screen content displayed on the external display to be out of sync with the audio. Summary of the Invention

[0004] This application provides a method for adjusting screen projection display parameters, which solves the problem that the projection content displayed on the display screen is out of sync with the audio when an external display screen is connected to the screen projection system, and improves user experience.

[0005] In a first aspect, the present application provides a screen projection system, which includes an electronic device, a large-screen device and a display device; the large-screen device is used to: establish a first screen projection connection with the electronic device; establish a second screen projection connection with the display device; the electronic device is used to send a first display parameter to the large-screen device; wherein the first display parameter includes a first encoding frame rate of the electronic device; the display device is used to send a second display parameter to the large-screen device; wherein the second display parameter includes a first decoding frame rate and a first display frame rate of the display device; the large-screen device is also used to: receive the first display parameter sent by the electronic device; receive the first display parameter sent by the display device The second display parameter of the large-screen device; obtaining the display parameter three of the large-screen device, the display parameter three including the second encoding frame rate, the second decoding frame rate and the second display frame rate; based on the first display parameter, the second display parameter and the display parameter three, determining the target display parameters, wherein the target display parameters include the target encoding frame rate, the target encoding frame rate is less than or equal to the minimum value of the first encoding frame rate, the first decoding frame rate, the first display frame rate, the second encoding frame rate, the second decoding frame rate and the second display frame rate; based on the target encoding frame rate in the target display parameters, encode the first projection content, and project the first projection content to the display device.

[0006] Through the system of the first aspect, when an external display screen is connected to the projection system, the newly formed projection system will negotiate the display parameters of each device in the projection system to prevent the problem of the projection content displayed on the display screen being out of sync with the audio when an external display screen is connected to the projection system, thereby improving user experience.

[0007] In combination with the system of the first aspect, in one possible implementation method, before the large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameters and projects the first projection content to the display device, the large-screen device is also used to: send the target display parameters to the electronic device, wherein the target display parameters are used to instruct the electronic device to encode the second projection content at the target encoding frame rate and send it to the large-screen device; the electronic device is also used to: encode the second projection content at the target encoding frame rate; send the second projection content to the large-screen device; the large-screen device is also used to: receive the second projection content sent by the electronic device; and display the second projection content.

[0008] In combination with the system of the first aspect, in one possible implementation method, the large-screen device is specifically used to: establish the second screen projection connection with the display device after the large-screen device establishes the first screen projection connection with the electronic device; or, establish the first screen projection connection with the electronic device after the large-screen device establishes the second screen projection connection with the display device.

[0009] In combination with the system of the first aspect, in one possible implementation, before the large-screen device receives the first display parameter sent by the electronic device, the large-screen device is also used to: send a first request to the electronic device, wherein the first request is used to instruct the electronic device to send the first display parameter to the large-screen device; the electronic device is also used to: receive the first request sent by the large-screen device; and send the first display parameter to the large-screen device in response to the first request; the large-screen device is also used to: receive the first display parameter sent by the electronic device.

[0010] In combination with the system of the first aspect, in one possible implementation, the large-screen device is specifically used to: when it is determined that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, send the first request to the electronic device.

[0011] In combination with the system of the first aspect, in one possible implementation, the first display parameter also includes a first encoding type and / or a first screen resolution; the second display parameter also includes a second encoding type and / or a second screen resolution; the target display parameter also includes a target encoding type and / or a target screen resolution; wherein the target encoding type is a coding type common to the first encoding type and the second encoding type, and the target screen resolution is a screen resolution common to the first screen resolution and the second screen resolution.

[0012] In a second aspect, the present application provides a method for adjusting screen projection display parameters, the method comprising: a large-screen device establishes a first screen projection connection with an electronic device, and the large-screen device establishes a second screen projection connection with a display device; the large-screen device receives a first display parameter sent by the electronic device, the first display parameter including a first encoding frame rate of the electronic device; the large-screen device receives a second display parameter sent by the display device, the second display parameter including a first decoding frame rate and a first display frame rate of the display device; the large-screen device obtains a third display parameter of the large-screen device, the third display parameter including a second encoding frame rate, a second decoding frame rate and a second display frame rate; the large-screen device determines a target display parameter based on the first display parameter, the second display parameter and the third display parameter, wherein the target display parameter includes a target encoding frame rate, and the target encoding frame rate is less than or equal to a minimum value among the first encoding frame rate, the first decoding frame rate, the first display frame rate, the second encoding frame rate, the second decoding frame rate and the second display frame rate;

[0013] The large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameters, and projects the first projection content to the display device.

[0014] Through the method of the first aspect, when an external display screen is connected to the projection system, the newly formed projection system will negotiate the display parameters of each device in the projection system to prevent the problem of the projection content displayed on the display screen being out of sync with the audio when an external display screen is connected to the projection system, thereby providing user experience.

[0015] In combination with the method of the first aspect, in a possible implementation, before the large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameters and projects the first projection content to the display device, the method also includes: the large-screen device sends the target display parameters to the electronic device, wherein the target display parameters are used to instruct the electronic device to encode the second projection content at the target encoding frame rate and send it to the large-screen device; the large-screen device receives the second projection content sent by the electronic device; and the large-screen device displays the second projection content.

[0016] In combination with the method of the first aspect, in a possible implementation, the large-screen device establishes a first screen projection connection with the electronic device, and the large-screen device establishes a second screen projection connection with the display device, specifically including: after the large-screen device establishes the first screen projection connection with the electronic device, the large-screen device establishes the second screen projection connection with the display device; or, after the large-screen device establishes the second screen projection connection with the display device, the large-screen device establishes the first screen projection connection with the electronic device.

[0017] In combination with the method of the first aspect, in a possible implementation, before the large-screen device receives the first display parameter sent by the electronic device, the method also includes: the large-screen device sends a first request to the electronic device, and the first request is used to instruct the electronic device to send the first display parameter to the large-screen device.

[0018] In combination with the method of the first aspect, in a possible implementation, the large-screen device sends a first request to the electronic device, specifically including: when the large-screen device determines that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, the large-screen device sends the first request to the electronic device.

[0019] In combination with the method of the first aspect, in a possible implementation, the first display parameter also includes a first encoding type and / or a first screen resolution; the second display parameter also includes a second encoding type and / or a second screen resolution; the target display parameter also includes a target encoding type and / or a target screen resolution; wherein the target encoding type is a coding type common to the first encoding type and the second encoding type, and the target screen resolution is a screen resolution common to the first screen resolution and the second screen resolution.

[0020] In a third aspect, the present application provides an electronic device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code. The computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the steps of the electronic device execution method in any possible implementation of any of the above aspects.

[0021] In a fourth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the steps of the electronic device execution method in any possible implementation of any of the above aspects.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, enables an electronic device to execute the steps of the electronic device execution method in any possible implementation of any of the above aspects.

[0023] In a sixth aspect, an embodiment of the present application provides a large-screen device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, which includes computer instructions. When the one or more processors execute the computer instructions, the large-screen device performs the steps of the large-screen device execution method in any possible implementation of any of the above aspects.

[0024] In a seventh aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on a large-screen device, the large-screen device executes the steps of the method executed by the large-screen device in any possible implementation of any of the above aspects.

[0025] In an eighth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables a large-screen device to execute the steps of the method executed by the large-screen device in any possible implementation of any of the above aspects.

[0026] In a ninth aspect, embodiments of the present application provide a display device comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are configured to store computer program code. The computer program code comprises computer instructions, and when the one or more processors execute the computer instructions, the display device executes the steps of the display device execution method in any possible implementation of any of the aforementioned aspects.

[0027] In a tenth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on a display device, enables the display device to execute the steps of the display device execution method in any possible implementation of any of the above aspects.

[0028] In an eleventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables a display device to execute the steps of the display device execution method in any possible implementation of any of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a system provided in an embodiment of the present application;

[0030] Figure 2A-2D A set of UI diagrams provided for the embodiments of this application;

[0031] Figure 2E-2F Another set of UI diagrams provided in the embodiment of the present application;

[0032] Figure 3 A flowchart of a method for negotiating display parameters between an electronic device 100 and a large-screen device 200 provided in an embodiment of the present application;

[0033] Figure 3A-3B A UI diagram of a group of electronic devices 100 playing screen projection content 1 provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a process in which a large-screen device 200 negotiates display parameters among an electronic device 100, the large-screen device 200, and a display device 300, provided in an embodiment of the present application;

[0035] Figure 4A An embodiment of the present application provides a UI diagram of an electronic device 100 displaying multimedia content;

[0036] Figure 5 A schematic diagram of an RTCP data packet format provided in an embodiment of the present application;

[0037] Figure 6 A schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0038] Figure 7 A software structure block diagram of an electronic device 100 provided in an embodiment of the present application;

[0039] Figure 8 The hardware structure of a large-screen device 200 provided in an embodiment of the present application;

[0040] Figure 9 This is a hardware structure of a display device 300 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following is a clear and detailed description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0042] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0043] The term "user interface (UI)" in the specification, claims and drawings of this application refers to the media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and Extensible Markup Language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as images, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, input boxes, buttons, scroll bars, images and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as <head> and <body>. A node corresponds to a control or attribute in the interface, and the node is presented as user-visible content after parsing and rendering. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also called a page, can be understood as a special control embedded in the application interface. A web page is a source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code, such as HTML through <body>. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0044] A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that uses graphics to display. It can be a window, control, or other interface element displayed on the display of an electronic device.

[0045] At present, the electronic device 100 (such as a mobile phone) can establish a projection connection with the large-screen device 200 (such as a personal computer), and then the electronic device 100 projects the projection content and displays it on the large-screen device 200. Before the electronic device 100 displays the projection content on the large-screen device 200, the electronic device 100 negotiates the projection parameters with the large-screen device 200. Specifically, the large-screen device 200 sends the display parameters to the electronic device 100. After receiving the display parameters sent by the large-screen device 200, the electronic device 100 negotiates with the display parameters of the large-screen device 200 to obtain the target display parameters after negotiation. Afterwards, the electronic device 100 processes the projection content based on the negotiated target display parameters and sends it to the large-screen device 200, and the large-screen device 200 receives and displays the projection content.

[0046] The negotiation of display parameters between the electronic device 100 and the large-screen device 200 will be described in detail in subsequent embodiments and will not be repeated in this application.

[0047] The screen projection content can be videos, pictures, audio or tables, etc.

[0048] The display parameters of the large-screen device 200 include any one or more of the following: screen resolution, display frame rate, decoding frame rate, encoding type, etc.

[0049] The display parameters of the electronic device 100 include any one or more of the following: screen resolution, encoding frame rate, encoding type, etc.

[0050] The target display parameters include any one or more of the following: screen resolution, encoding type, encoding frame rate, etc.

[0051] However, when the large-screen device 200 is connected to the display device 300, the large-screen device 200 processes the projected content according to the display parameters of the large-screen device 200 and sends it to the display device 300. If the display frame rate or decoding frame rate of the display device 300 is lower than the encoding frame rate of the large-screen device 200, the image frames on the display device 300 side will continue to accumulate, resulting in a freeze on the display device 300 side; or because the central processing unit (CPU) of the large-screen device 200 has limited capabilities, the encoding frame rate of the large-screen device 200 is lower than the display frame rate of the large-screen device 200. Therefore, when the large-screen device 200 processes the projected content according to the encoding frame rate, the image frames on the large-screen device 200 side continue to accumulate. Before the large-screen device 200 sends the processed projected content to the display device 300, the image frames accumulate on the large-screen device 200 side, resulting in the image frames displayed on the display device 300 side being out of sync with the audio played by the large-screen device 200, affecting the user experience.

[0052] Therefore, the following embodiment of the present application provides a method for adjusting screen projection display parameters. The method includes: the large-screen device establishes a first screen projection connection with the electronic device, and the electronic device establishes a second screen projection connection with the display device. The large-screen device will negotiate the target display parameters based on the display parameters of the large-screen device, the display parameters of the electronic device, and the display parameters of the display device. The target display parameters include a target encoding frame rate, so that the target encoding frame rate is less than the display frame rate or decoding frame rate of the display device, or the display frame rate of the large-screen device 200 is greater than the target encoding frame rate. Afterwards, the large-screen device will process the projection content according to the target display parameters and send it to the display device. In this way, the problem that the projection content displayed on the display screen is out of sync with the audio when an external display screen is connected to the projection system is solved, and user experience is improved.

[0053] The screen projection connection can be a collaborative screen projection connection, a mirrored screen projection connection, etc. This application does not limit the method of screen projection connection. The following embodiments of this application are described using collaborative screen projection connection as an example.

[0054] Next, a system architecture provided by an embodiment of the present application is introduced.

[0055] like Figure 1 As shown, Figure 1 A schematic diagram of a system provided in an embodiment of the present application.

[0056] First, the electronic device 100 establishes a screen projection connection with the large-screen device 200, that is, the electronic device 100 can project the screen content and display it on the large-screen device 200. Afterwards, the large-screen device 200 can be connected to the external display device 300 through a USB interface or other means. In this way, the large-screen device 200 displays the display content of the large-screen device 200 on the display device 300.

[0057] In some embodiments, the display device 300 may be connected to the large-screen device 200 through a USB interface only after the electronic device 100 establishes a screen projection connection with the large-screen device 200; in other embodiments, the display device 300 may be connected to the large-screen device 200 through a USB interface before the electronic device 100 establishes a screen projection connection with the large-screen device 200.

[0058] The electronic device 100 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, vehicle-mounted device, smart home device and / or smart city device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device. The software system of the electronic device 100 includes but is not limited to Linux or other operating systems. It is Huawei's Hongmeng system.

[0059] The large-screen device 200 refers to an electronic device with a relatively large display screen, such as a television, a desktop computer, or an electronic billboard.

[0060] The display device 300 is an image (eg, video, picture, etc.) output device, which can be used to display images (eg, video, picture, etc.).

[0061] When the electronic device 100 establishes a screen projection connection with the large-screen device 200, the electronic device 100 and the large-screen device 200 can be connected and communicated through wireless communication technology. The wireless communication technology here includes but is not limited to: wireless local area network (WLAN) technology, Bluetooth, infrared, near-field communication (NFC), ZigBee, and other wireless communication technologies that appear in subsequent developments. For the sake of convenience of description, the following embodiments will be described by taking the communication between the electronic device 100 and the large-screen device 200 through wireless fidelity direct (Wi-Fi direct) (also known as wireless fidelity peer-to-peer (Wi-Fi P2P)) technology as an example.

[0062] The electronic device 100 can join the WiFi P2P group constructed by the large-screen display device 200 through Wi-Fi P2P technology and connect to the large-screen device 200. After that, the electronic device 100 sends synchronization information (such as handshake information) to the large-screen device 200 to perform network synchronization. After the network is successfully established and the synchronization is completed, the large-screen device 200 in the collaborative network performs collaborative playback, collaborative recording, collaborative conferencing, etc. under the control of the electronic device 100. That is, the electronic device 100 sends the projection content to the large-screen device 200 through the established Wi-Fi P2P connection, so that the large-screen device 200 displays and / or plays the projection content, thereby completing the projection. The projection content may include but is not limited to images (such as videos, pictures, etc.) and audio.

[0063] In some embodiments, the display device 300 may be connected to the large-screen device 200 through a USB interface only after the electronic device 100 establishes a screen projection connection with the large-screen device 200.

[0064] When the electronic device 100 establishes a screen projection connection with the large-screen device 200, the display parameters are negotiated to obtain the target display parameters. The electronic device 100 and the large-screen device 200 perform collaborative screen projection using the target display parameters. The negotiation of display parameters between the electronic device 100 and the large-screen device 200 will be described in detail in subsequent embodiments and will not be repeated in this application.

[0065] After the electronic device 100 and the large-screen device 200 establish collaborative screen projection, the large-screen device 200 is connected to the external display device 300 through the USB interface, etc., to ensure that when the large-screen device 200 displays the projection content on the display device 300, the projection content displayed on the display device 300 side will not be stuck and the projection content displayed on the display device 300 side is synchronized with the sound on the large-screen device 200 side. Therefore, when the large-screen device 200 is connected to the external display device 300, the large-screen device 200 (or the electronic device 100) will negotiate the display parameters of the electronic device 100, the large-screen device 200 and the display device 300 to obtain the target display parameters. Afterwards, the electronic device 100 will process the projection content according to the target display parameters and send it to the large-screen device 200. The large-screen device 200 will also process the projection content according to the target display parameters and send it to the large-screen device display device 300. The large-screen device 200 negotiates the display parameters of the electronic device 100, the large-screen device 200 and the display device 300 to obtain the target display parameters, which will be described in detail in subsequent embodiments and will not be repeated in this application.

[0066] In other embodiments, before the electronic device 100 establishes a screen projection connection with the large-screen device 200, the display device 300 establishes a connection with the large-screen device 200 through a USB interface.

[0067] In order to ensure that when the large-screen device 200 displays the projected content on the display device 300, the projected content displayed on the display device 300 side will not be stuck and the projected content displayed on the display device 300 side is synchronized with the sound on the large-screen device 200 side. Therefore, when the electronic device 100 and the large-screen device 200 establish collaborative screen projection, the large-screen device 200 (or the electronic device 100) will negotiate the display parameters of the electronic device 100, the large-screen device 200 and the display device 300 to obtain the target display parameters. Afterwards, the electronic device 100 will process the projected content according to the target display parameters and send it to the large-screen device 200. The large-screen device 200 will also process the projected content according to the target display parameters and send it to the large-screen device display device 300. The large-screen device 200 negotiates the display parameters of the electronic device 100, the large-screen device 200 and the display device 300 to obtain the target display parameters, which will be introduced in detail in subsequent embodiments and will not be repeated in this application.

[0068] The following describes in detail how the electronic device 100 and the large-screen device 200 establish a screen projection connection with reference to the UI diagram.

[0069] The electronic device 100 can establish a screen projection connection with the large-screen device 200 in any of the following ways.

[0070] Method 1:

[0071] Figure 2A-2D The example shows the user operation of turning on the collaborative screen projection function detected by the electronic device 100.

[0072] Figure 2A An exemplary video playback user interface 20 on a mobile device 100 is shown. The user interface 20 displays: the video title (Brave Step), the video playback progress (Episode 1), the video playback duration (4 minutes and 13 seconds), the total video duration (7 minutes and 33 seconds), the popularity of the video (5946), the total number of video episodes (23 episodes in total), more video icons, and images corresponding to more videos.

[0073] like Figure 2A and Figure 2B As shown, when the electronic device 100 detects a downward sliding gesture on the display screen, in response to the sliding gesture, the mobile device 100 displays a window 201 on the user interface 20. Figure 2B As shown, a control 202 may be displayed in window 201, and control 202 may receive an operation (such as a touch operation, a click operation) to turn on / off the collaborative screen projection function of the mobile device 100. The expression form of control 202 may include an icon and / or text (such as the text "collaborative screen projection"). Other functions such as Wi-Fi, Bluetooth, flashlight, ringing, automatic rotation, instant sharing, flight mode, mobile data, location information, screenshot, eye protection mode, hotspot, screen recording, NFC and other switch controls may also be displayed in window 201, that is, a user operation to turn on the collaborative screen projection function is detected. In some embodiments, after the electronic device 100 detects a user operation acting on the control 202, it can change the display form of the control 202, such as adding a shadow when displaying the control 202.

[0074] Not limited to Figure 2A On the main interface shown, the user can also input a downward sliding gesture on other interfaces to trigger the electronic device 100 to display window 201.

[0075] Not limited to Figure 2A and Figure 2B The user operation shown is the user operation on the control 202 in the window 201. In the embodiment of the present application, the user operation of turning on the collaborative screen projection function can also be implemented in other forms, which is not limited by the embodiment of the present application.

[0076] For example, the electronic device 100 can also display a settings interface provided by a settings application, which may include controls provided to the user for turning on / off the collaborative screen projection function of the electronic device 100. The user can turn on the collaborative screen projection function of the electronic device 100 by inputting user operations on the control.

[0077] Upon detecting a user operation to enable the collaborative screen projection function, the electronic device 100 enables one or more of Wi-Fi Direct (not shown), Bluetooth, or NFC in the wireless communication module 160, and uses one or more of Wi-Fi Direct, Bluetooth, and NFC to discover nearby electronic devices capable of screen projection. For example, the electronic device 100 can discover nearby large-screen devices 200 and other electronic devices through Wi-Fi Direct.

[0078] In addition to displaying the identification of the electronic devices discovered by the electronic device 100 that are acceptable for collaborative screen projection, the electronic device 100 can also display other information, such as images of the discovered electronic devices, etc., which is not limited in the embodiments of the present application.

[0079] Then, illustratively, as Figure 2C As shown, a window 203 pops up on the electronic device 100. The window 203 includes: an interface indicator 204, an icon 205, an image 206 of one or more electronic devices, and a logo 207.

[0080] For example, Figure 2C As shown, the user operation of selecting the large-screen device 200 may be a user operation acting on the image 206 and / or logo 207 corresponding to the large-screen device 200. The user operation of selecting the large-screen device 200 may also be implemented in other forms, which are not limited in the embodiment of the present application.

[0081] In response to the user operation, the electronic device 100 can establish a communication connection with the large-screen device 200 through one or more wireless communication technologies such as Wi-Fi Direct, Bluetooth, and NFC. For example, the electronic device 100 and the large-screen device 200 establish a Wi-Fi Direct communication connection. After the electronic device 100 and the large-screen device 200 establish a communication connection, they can conduct capability negotiation based on the communication connection, including the encoding format, resolution, audio format, encoding frame rate, display frame rate, decoding frame rate, etc. supported by both parties, to facilitate the subsequent transmission of the projection content.

[0082] Specifically, the electronic device 100 can obtain the currently displayed multimedia content (including images and / or audio) by recording the screen or recording the audio, and then compress the obtained content and send it to the large-screen device 200 through the communication connection between the electronic device 100 and the large-screen device 200. Taking the electronic device 100 and the large-screen device 200 sharing multimedia content based on miracast as an example, the electronic device 100 can obtain the image displayed on the display screen by recording the screen according to the provisions of the miracast protocol, and compress the image using the H.264 encoding algorithm; collect the audio played by the electronic device 100, and compress the audio using the advanced audio coding (AAC) algorithm; then encapsulate the compressed audio data and image data into a transport stream (TS), and then encode the TS stream according to the real-time transport protocol (RTP) and send the encoded data to the large-screen device 200 via a Wi-Fi direct connection. That is, the multimedia content is transmitted in a streaming media manner.

[0083] like Figure 2D As shown, the electronic device 100 transmits the currently displayed multimedia content (including images and / or audio) to the large-screen device 200. The large-screen device 200 receives the currently displayed multimedia content (including images and / or audio) transmitted by the electronic device 100 and displays it on the display screen of the large-screen device 200.

[0084] After receiving the multimedia content sent by the electronic device 100 based on the communication connection, the large-screen device 200 can perform decoding processing on the multimedia content to obtain the multimedia content. Taking the electronic device 100 and the large-screen device 200 sharing multimedia content based on miracast as an example, the large-screen device 200 can receive the RTP-encoded TS stream based on the Wi-Fi direct communication connection between the large-screen device 200 and the electronic device 100, and can perform RTP decoding, TS stream decapsulation, audio and video quality processing / delay synchronization processing on it in sequence, and finally output the audio and video, that is, play the multimedia content.

[0085] Method 2:

[0086] Figure 2E-2F The example shows that the electronic device 100 is close to the NFC sensing area of the large-screen device 200, triggering the electronic device 100 to start the collaborative screen projection function.

[0087] like Figure 2E As shown, the electronic device 100 is close to the NFC sensing area of the large-screen device 200, and the large-screen device 200 receives and responds to the connection request sent by the electronic device 100, and the large-screen device 200 displays the following on the display screen: Figure 2F Prompt box 208 is shown.

[0088] The prompt box 208 includes a prompt message “Do you want to allow HUA WEIP40 to connect to this computer?” “HUA WEIP40” is the device identifier of the electronic device 100.

[0089] The prompt box 208 also includes an allow control 209 and a reject control 210. The allow control 209 can receive an input operation (e.g., a single click), and in response to the input operation (e.g., a single click), the large-screen device 200 establishes a communication connection with the electronic device 100 via NFC. The reject control 210 can receive an input operation (e.g., a single click), and in response to the input operation (e.g., a single click), the large-screen device 200 rejects establishing a communication connection with the electronic device 100.

[0090] Exemplarily, when the control 209 is allowed to receive an input operation (e.g., a click), in response to the input operation (e.g., a click), the large-screen device 200 establishes a communication connection with the electronic device 100 via NFC. After the electronic device 100 and the large-screen device 200 establish a communication connection, they can perform capability negotiation based on the communication connection, including the encoding format, resolution, audio format, encoding frame rate, display frame rate, decoding frame rate, etc. supported by both parties, to facilitate the subsequent transmission of the projection content.

[0091] Specifically, the electronic device 100 can obtain the currently displayed multimedia content (including images and / or audio) by recording the screen or recording the audio, and then compress the obtained content and send it to the large-screen device 200 through the communication connection between the electronic device 100 and the large-screen device 200. Taking the electronic device 100 and the large-screen device 200 sharing multimedia content based on miracast as an example, the electronic device 100 can obtain the image displayed on the display screen by recording the screen according to the provisions of the miracast protocol, and compress the image using the H.264 encoding algorithm; collect the audio played by the electronic device 100, and compress the audio using the advanced audio coding (AAC) algorithm; then encapsulate the compressed audio data and image data into a transport stream (TS), and then encode the TS stream according to the real-time transport protocol (RTP) and send the encoded data to the large-screen device 200 via a Wi-Fi direct connection. That is, the multimedia content is transmitted in a streaming media manner.

[0092] The electronic device 100 transmits the currently displayed multimedia content (including images and / or audio) to the large-screen device 200. The large-screen device 200 receives the currently displayed multimedia content (including images and / or audio) transmitted by the electronic device 100 and displays it on the display screen of the large-screen device 200. For details, please refer to the above Figure 2D The embodiments shown in this application will not be described in detail here.

[0093] Next, a process of negotiating display parameters between the electronic device 100 and the large-screen device 200 after the electronic device 100 and the large-screen device 200 establish a communication connection via WiFi P2P will be described.

[0094] like Figure 3 As shown, Figure 3 The flowchart of the method for the electronic device 100 to negotiate display parameters with the large-screen device 200 is exemplarily shown.

[0095] S301. The electronic device 100 obtains projection content 1, where the projection content 1 includes a first image or the projection content 1 includes a first image and a first audio.

[0096] The electronic device 100 can obtain the projection content 1 by recording the screen and / or audio.

[0097] The screen content can be as follows Figure 2D The electronic device 100 is shown displaying multimedia content (including a first image or a first image and a first audio).

[0098] The first projection content is the content displayed in real time by the electronic device 100.

[0099] It should be noted that the projection content 1 displayed by the electronic device 100 may also only include the first image, and this application does not limit the form of the projection content 1.

[0100] Figure 3A-3B The UI diagram of the electronic device 100 playing the projection content 1 is shown as an example.

[0101] Figure 3A The user interface 30 of the electronic device 100 is shown as an example. The user interface 30 may include icons of some applications. For example, an icon 301 for file management, an icon 302 for email, an icon 303 for music, an icon 304 for Huawei video, an icon 305 for sports and health, an icon 306 for weather, an icon 307 for camera, an icon 308 for address book, an icon 309 for phone, and an icon 310 for information. In some embodiments, the user interface 30 may include icons of more or fewer applications. In some embodiments, the user interface 30 may include icons of some applications related to Figure 3A The icons of the applications shown are different from each other and are not limited here.

[0102] The electronic device 100 may launch the video application in response to a user operation on the Huawei Video icon 304 on the user interface 30 . Figure 3B The user interface 40 displayed after the electronic device 100 starts the video application is shown as an example. The user interface 40 is the main page provided by the Huawei video application. Figure 3B As shown, one or more video images 401 are displayed in the user interface 40. The video image can be dynamic or static. In addition, the user interface 40 can also display a bottom menu bar, a search box, a sub-channel entrance, etc., which is not limited in this embodiment of the present application. Figure 3B As shown, the electronic device 100 can detect the user operation on the video image 401, obtain the network video indicated by the video image 401 from the server corresponding to the Huawei video application through the network, and play the network video. The network video indicated by the video image 401 that monitors the user operation is the network video selected by the user. The electronic device 100 will display Figure 2A The video player user interface 20 shown. The description of the video player user interface 20 can refer to Figure 2A The description of , this application will not be repeated here.

[0103] S302. The electronic device 100 establishes a screen projection connection with the large-screen device 200.

[0104] The process of establishing a connection between the electronic device 100 and the large screen device 200 is described in detail in the following. Figure 2A-2D and Figure 2E-2F The embodiments shown in this application will not be described in detail here.

[0105] After the electronic device 100 establishes a connection with the large-screen device 200, the electronic device 100 negotiates display parameters with the large-screen device 200. For details, please refer to the description of S303 and S304.

[0106] S303. The electronic device 100 receives a message 1 sent by the large-screen device 200. The message 1 includes a display parameter 1. The display parameter 1 includes a decoding frame rate 1 and a display frame rate 1.

[0107] Display frame rate 1 is the number of image frames that the large-screen device 200 can display within a fixed time (eg, 1 s).

[0108] The decoding frame rate 1 is the number of image frames transmitted by the electronic device 100 that the large-screen device 200 can decode within a fixed time (for example, 1 second).

[0109] Display parameter 1 also includes other parameters, such as screen resolution 1 and encoding type 1. Display parameter 1 may also include other parameters, which are not limited in this application.

[0110] Screen resolution 1 is the number of pixels displayed in the horizontal and vertical directions of the large-screen device 200. For example, screen resolution 1 is 150×128, which means that the number of pixels in the horizontal direction is 150 and the number of pixels in the vertical direction is 128.

[0111] Coding type 1 includes a speech coding type and an image stream coding type. For example, the image stream coding type includes any of the following: Xvid, AVC / H.264, MPEG1, and MPEG2; the speech coding type includes any of the following: MP3 and AAC. It should be noted that the image stream coding type and speech coding type can also be other types, and this application does not limit them.

[0112] S304: The electronic device 100 obtains target display parameter 1 according to display parameter 2 and display parameter 1. Target display parameter 1 includes encoding frame rate 1. Encoding frame rate 2 is less than the minimum value of frame rate parameter values of display parameter 1 and display parameter 2.

[0113] The second display parameter includes a second encoding frame rate of the electronic device 100. The second encoding frame rate is the number of audio data frames encoded by the electronic device 100 per second.

[0114] Display parameter 2 also includes other parameters, such as screen resolution 2 and encoding type 2. Display parameter 2 may also include other parameters, which are not limited in this application.

[0115] After the electronic device 100 receives the display parameter 1 of the large-screen device 200, the electronic device 100 will compare the display frame rate 1, the decoding frame rate 1 in the display parameter 1 and the encoding frame rate 2 in the display parameter 2, obtain the minimum value of the three, and determine the encoding frame rate 1 based on the minimum value of the three, where the encoding frame rate 1 is less than or equal to the minimum value of the display frame rate 1, the decoding frame rate 1 and the encoding frame rate 2.

[0116] The electronic device 100 will also negotiate other parameters in the display parameter 1 and the display parameter 2, so that the parameter information of other display parameters of the electronic device 100 and the electronic device 200 are consistent.

[0117] Exemplarily, the electronic device 100 determines screen resolution 2 in display parameter 2, compares the sizes of screen resolution 2 and screen resolution 1, obtains the minimum value between the two, and determines screen resolution 3 based on the minimum value between the two, wherein screen resolution 3 is less than or equal to the minimum value between screen resolution 2 and screen resolution 1.

[0118] Exemplarily, the electronic device 100 will also determine the encoding type 2 in the display parameter 2, and the electronic device 100 determines the common encoding type of encoding type 2 and encoding type 1, as the encoding type 3 obtained by negotiation between the electronic device 100 and the electronic device 200. The encoding type 3 is any one of the common encoding types of encoding type 2 and encoding type 1.

[0119] The electronic device 100 will also negotiate other parameters in the display parameter 1 and the display parameter 2, which will not be introduced one by one here.

[0120] Therefore, in addition to encoding frame rate 1, target display parameter 1 also includes encoding type 3 and screen resolution 3. Target display parameter 1 may also include other parameters, which will not be introduced one by one in this application.

[0121] S305 . The electronic device 100 sends a second message to the large-screen device 200 . The second message includes the first target display parameter.

[0122] The target display parameter 1 includes encoding frame rate 1. The target display parameter 1 may also include screen resolution 3 and encoding type 3.

[0123] The electronic device 100 sends a second message to the large screen device 200 , where the second message includes the target display parameter 1. In response to the second message sent by the electronic device 100 to the large screen device 200 , the large screen device 200 receives the second message sent by the electronic device 100 to the large screen device 200 .

[0124] S306. The electronic device 100 processes the projection content one with the target display parameter one.

[0125] S307. The electronic device 100 sends the projection content processed with the target display parameter to the large-screen device 200.

[0126] The electronic device 100 can obtain the projection content one by recording the screen and / or audio, and process the projection content one according to the target display parameter one to obtain the projection content one processed with the target display parameter one.

[0127] The electronic device 100 compresses the projection content processed with the target display parameter one, and then sends the projection content processed with the target display parameter one to the large-screen device 200.

[0128] S308. The large-screen device 200 receives the projection content 1 processed with the target display parameter 1, and displays the projection content 1.

[0129] In response to the projection content 1 processed with the target display parameter 1 sent by the electronic device 100, the large screen device 200 receives the projection content 1 processed with the target display parameter 1, and decodes the projection content 1 processed with the target display parameter 1 in sequence at the second decoding rate, processes the audio and video quality, and then displays the projection content 1. Figure 2D As shown, Figure 2D Schematic diagram of large-screen device 200 displaying projection content 1. The value of the second decoding rate is consistent with the value of the encoding frame rate 1.

[0130] After the electronic device 100 establishes a screen projection connection with the large-screen device 200, the electronic device 100 projects the projected content and displays it on the large-screen device 200. Subsequently, the large-screen device 200 can connect to an external display device 300 via a USB interface or other means. The large-screen device 200 negotiates display parameters among the electronic device 100, large-screen device 200, and display device 300. The following describes in detail the process by which the large-screen device 200 negotiates display parameters among the electronic device 100, large-screen device 200, and display device 300.

[0131] In some embodiments, the electronic device 100 may also negotiate the display parameters of the electronic device 100, the large-screen device 200, and the display device 300. This application does not limit the device for negotiating display parameters. The following embodiments of this application use the large-screen device 200 negotiating the display parameters of the electronic device 100, the large-screen device 200, and the display device 300 as an example for detailed description.

[0132] like Figure 4 As shown, Figure 4 The diagram exemplarily shows a process in which the large-screen device 200 negotiates display parameters between the electronic device 100 , the large-screen device 200 and the display device 300 .

[0133] S401. The electronic device 100 establishes a screen projection connection (first screen projection connection) with the large-screen device 200.

[0134] The process of establishing a screen projection connection between the electronic device 100 and the large screen device 200 can be found in Figure 2A-2D and Figure 2E-2F The embodiments shown in this application will not be described in detail here.

[0135] After the electronic device 100 establishes a screen projection connection with the large screen device 200, the electronic device 100 and the large screen device 200 will negotiate display parameters. After the electronic device 100 and the large screen device 200 negotiate display parameters, the target display parameter 1 is obtained. The target display parameter 1 includes encoding frame rate 2, third screen resolution, encoding type 3, etc. Please refer to the process of the electronic device 100 and the large screen device 200 negotiating display parameters. Figure 3 The embodiments shown are not described in detail in this application.

[0136] S402. The large device 200 receives the first operation, and the large-screen device 200 establishes a connection with the display device 300 (a second screen projection connection).

[0137] The large-screen device 200 can be connected to the display device 300 via a USB interface or the like, and the large-screen device 200 establishes a connection with the display device 300 via a USB data cable.

[0138] In some embodiments, before the large-screen device 200 establishes a screen projection connection with the display device 300, the large-screen device 200 has already established a connection with the display device 300. That is, S402 may not exist.

[0139] S403 , the electronic device 100 sends a second message to the large-screen device 200 , where the second message displays a second parameter (a first display parameter).

[0140] The second display parameter includes a second encoding frame rate (first encoding frame rate) of the electronic device 100. The first display parameter may also include a second screen resolution (first screen resolution) and a second encoding type (first encoding type).

[0141] S404. The display device 300 sends message three to the large-screen device 200. Message three includes display parameter three (second display parameter). Display parameter three includes display frame rate three (first display frame rate) and decoding frame rate three (first decoding frame rate) of the display device 300.

[0142] Display frame rate three is the number of image frames that the display device 300 can display within a fixed time (eg, 1 s).

[0143] The decoding frame rate three is the number of image frames transmitted by the large-screen device 200 that the display device 300 can decode within a fixed time (eg, 1 s).

[0144] Display parameter three also includes other parameters, such as screen resolution four (second screen resolution) and encoding type four (second encoding type). Display parameter three may also include other parameters, which are not limited in this application.

[0145] Screen resolution 4 is the number of pixels displayed in the horizontal and vertical directions of the display device 300. For example, screen resolution 4 is 160×128, which means that the number of pixels in the horizontal direction is 160 and the number of pixels in the vertical direction is 128.

[0146] Coding type four includes a voice coding type and an image stream coding type. For example, the image stream coding type includes any of the following: Xvid, AVC / H.264, MPEG1, and MPEG2; the voice coding type includes any of the following: MP3 and AAC. It should be noted that the image stream coding type and voice coding type can also be other types, such as image quality and color, which are not limited in this application.

[0147] In some embodiments, before the large-screen device 200 establishes a screen projection connection with the display device 300 , the large-screen device 200 has already established a connection with the display device 300 , and the electronic device 100 needs to send the display parameter 2 of the electronic device 100 to the large-screen device 200 .

[0148] S405. The large-screen device 200 obtains the second target display parameter through negotiation based on the display parameter one (the third display parameter), the display parameter two, and the display parameter three.

[0149] After the large-screen device 200 receives the display parameter 2 sent by the electronic device 100 and the display parameter 3 sent by the display device 300 , the large-screen device 200 will obtain the display parameter 1 of the large-screen device 200 at the same time.

[0150] The display parameter 1 includes the encoding frame rate 3 (third encoding frame rate), the decoding frame rate 1 (second decoding frame rate), and the display frame rate 1 (second display frame rate) of the large-screen device 200. The display parameter 1 may also include the screen resolution 1 (third screen resolution) and the encoding type 1 (third encoding type).

[0151] The second display parameter includes a second encoding frame rate of the electronic device 100. The first display parameter may also include a second screen resolution and a second encoding type.

[0152] The third display parameter includes a second display frame rate and a second decoding frame rate of the display device 300. The third display parameter also includes a fourth encoding type and a fourth screen resolution of the display device 300.

[0153] The large-screen device 200 will compare the encoding frame rate three, decoding frame rate one, and display frame rate one in display parameter one with the encoding frame rate two in display parameter two and the display frame rate two and decoding frame rate two in display parameter three to obtain the minimum value of the five frame rate parameters, and determine the encoding frame rate four (target encoding frame rate) based on the minimum value of the five frame rate parameters, where the encoding frame rate four is less than or equal to the minimum value of the encoding frame rate three, decoding frame rate one, display frame rate one, encoding frame rate two, display frame rate two, and decoding frame rate two.

[0154] The large-screen device 200 will also negotiate other parameters in display parameter 1, display parameter 2 and the third target display parameter so that the parameter information of other display parameters of the electronic device 100, the large-screen device 200 and the display device 300 are consistent.

[0155] Exemplarily, the large-screen device 200 determines the sizes of screen resolution one, screen resolution two, and screen resolution four in display parameter three, obtains the minimum value of the three, and determines screen resolution five (target screen resolution) based on the minimum value of the three, wherein screen resolution five is less than or equal to the minimum value of screen resolution one, screen resolution two, and screen resolution four.

[0156] Exemplarily, the large-screen device 200 will also determine the common coding type among coding type one, coding type two and coding type four, as coding type five (target coding type) obtained through negotiation between the electronic device 100, the large-screen device 200 and the display device 300. Coding type five is any one of the common coding types among coding type one, coding type two and coding type four.

[0157] The large-screen device 200 will also negotiate other parameters in display parameter 1, display parameter 2 and display parameter 3, which will not be introduced one by one in this application.

[0158] Therefore, the second target display parameter (target display parameter) includes not only the encoding frame rate four, but also the encoding type five and the screen resolution five. The second target display parameter may also include other parameters, which will not be introduced one by one in this application.

[0159] S406: The large-screen device 200 sends a fourth message to the electronic device 100, where the fourth message includes the second target display parameter.

[0160] In some embodiments, message four can also be called the first newly added extended RTCP APP message, which contains key projection factor adjustment information. The key projection factor adjustment information can be the second target display parameter, such as encoding frame rate four, encoding type five and screen resolution five.

[0161] S407 . The large-screen device 200 sends a fifth message to the display device 300 . The fifth message includes the second target display parameter.

[0162] The second target display parameter includes an encoding frame rate of 4. The first target display parameter may further include a screen resolution of 5 and an encoding type of 5.

[0163] The large screen device 200 sends a message 4 to the electronic device 100 , where the message 4 includes the second target display parameter. In response to the message 4 sent by the large screen device 200 to the electronic device 100 , the electronic device 100 receives the message 4 sent by the large screen device 200 .

[0164] At the same time, the large screen device 200 sends a fifth message including the second target display parameter to the display device 300. In response to the fifth message sent by the large screen device 200 to the display device 300, the display device 300 receives the fifth message sent by the large screen device 200.

[0165] In some embodiments, the large-screen device 200 may not send the message 5 to the display device 300, that is, S407 may not exist. This application does not limit this.

[0166] In some embodiments, before the large-screen device 200 receives the first display parameter sent by the electronic device 100, the large-screen device 200 sends a first request to the electronic device 100, where the first request is used to instruct the electronic device 100 to send the first display parameter to the large-screen device.

[0167] In some embodiments, the message including the first request may also be referred to as a second newly added extended RTCP APP message, which includes key projection factor adjustment information. The key projection factor adjustment information may be a first request, and the first request is used to instruct the electronic device 100 to send the first display parameter to the large-screen device.

[0168] In some embodiments, when the large-screen device 200 determines that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, the large-screen device sends a first request to the electronic device.

[0169] That is, the large-screen device 100 determines that the current display parameters no longer meet the load requirements, and the load will cause the display image and audio to be out of sync, and the large-screen device 200 will send a first request to the electronic device 100.

[0170] S408. The electronic device 100 obtains projection content 1, where the projection content 1 includes the first image or the projection content 1 includes the first image and the first audio.

[0171] In some embodiments, the electronic device 100 can obtain the projection content 1 by recording the screen and / or audio.

[0172] The screen content can be as follows Figure 2D The electronic device 100 is shown displaying multimedia content (including a first image or a first image and a first audio).

[0173] The first projection content is the content displayed in real time by the electronic device 100.

[0174] It should be noted that the projection content 1 displayed by the electronic device 100 may also only include the first image, and this application does not limit the form of the projection content 1.

[0175] S409. The electronic device 100 processes the projection content 1 (the second projection content) with the second target display parameter (target display parameter).

[0176] S410. The electronic device 100 sends the projection content processed with the second target display parameter to the large-screen device 200.

[0177] The electronic device 100 can obtain the projection content one by recording the screen and / or audio, and process the projection content one according to the second target display parameter to obtain the projection content one processed with the second target display parameter.

[0178] The electronic device 100 compresses the projection content 1 processed with the second target display parameter, and then sends the projection content 1 processed with the first target display parameter to the large-screen device 200.

[0179] S411. The large-screen device 200 receives the projection content 1 processed with the second target display parameter, and displays the projection content 1.

[0180] In response to the projection content 1 processed with the second target display parameters sent by the electronic device 100, the large screen device 200 receives the projection content 1 processed with the second target display parameters, and decodes the projection content 1 processed with the second target display parameters in sequence at the fourth decoding frame rate, processes the audio and video quality, and then displays the projection content 1. Figure 2D As shown, Figure 2D Schematic diagram of a large-screen device 200 displaying projection content 1. The value of the fourth decoding frame rate is the same as the value of the fourth encoding frame rate.

[0181] S412. The large-screen device 200 obtains the second projection content (the first projection content) by recording the screen and / or recording the audio. The second projection content includes the second image or the second projection content includes the second image and the second audio.

[0182] The second screen content can be as follows Figure 4A The electronic device 100 is shown displaying multimedia content (including images and / or audio).

[0183] The second image is the content displayed in real time by the large-screen device 200.

[0184] It should be noted that the projection content 2 displayed by the large-screen device 200 may also only include the second audio, and this application does not limit the form of the projection content 2.

[0185] S413. The large-screen device 200 sends the second projection content processed with the second target display parameters.

[0186] The large-screen device 200 compresses the projection content 2 processed with the second target display parameters, and then sends the projection content 2 processed with the second target display parameters to the display device 300.

[0187] S414. The display device 300 receives the second projection content processed with the second target display parameter and displays the second projection content.

[0188] In response to the projection content 2 processed with the second target display parameter sent by the large screen device 200, the display device 300 receives the projection content 2 processed with the second target display parameter, and decodes the projection content 2 processed with the second target display parameter in sequence at the fourth decoding frame rate, processes the audio and video quality, and then displays the projection content 2. Figure 4A As shown, Figure 4A Schematic diagram of display device 300 displaying projection content 2. The value of the fourth decoding frame rate is the same as the value of the fourth encoding frame rate.

[0189] It should be noted that Figure 4 The method is that the large-screen device 200 negotiates the target display parameters based on the first display parameters of the electronic device 100, the second display parameters of the display device 300, and the third display parameters of the large-screen device 200. In the embodiment of the present application, the target display parameters can also be obtained by the electronic device 100 negotiating the first display parameters of the electronic device 100, the second display parameters of the display device 300, and the third display parameters of the large-screen device 200. Specifically, when the large-screen device 200 determines that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, the electronic device 100 sends the second display parameters of the display device 300 and the third display parameters of the large-screen device 200 to the electronic device 100, and the electronic device 100 negotiates the target display parameters based on the first display parameters of the electronic device 100, the second display parameters of the display device 300, and the third display parameters of the large-screen device 200. The electronic device 100 obtains the target display parameters through negotiation based on the first display parameters of the electronic device 100, the second display parameters of the display device 300, and the third display parameters of the large-screen device 200. This is consistent with the method by which the large-screen device 200 obtains the target display parameters through negotiation based on the first display parameters of the electronic device 100, the second display parameters of the display device 300, and the third display parameters of the large-screen device 200, and this application will not repeat them here.

[0190] Next, in conjunction with a specific extended protocol, the display parameter negotiation between the electronic device 100, the large-screen device 200, and the display device 300 through the extended protocol will be introduced.

[0191] The implementation of the extended protocol involved in this application is to expand the quality of the screen projection service and its key influencing factor capabilities on the basis of the existing RTCP protocol (which only includes transmission quality such as the number of packet losses, the number of transmissions, etc.).

[0192] like Figure 5 As shown, Figure 5 This is a schematic diagram of the RTCP data packet format involved in the embodiments of the present application.

[0193] As shown in Table 1, Table 1 shows an exemplary Figure 5 The following table shows the fields contained in the RTCP packet and the meaning of each field.

[0194] Table 1

[0195]

[0196]

[0197] As shown in Table 1, the field version (version, V) indicates that the version number of the current protocol is 2, and the size of the field V is 2 bits. This application does not improve or expand the field V.

[0198] The padding field (P) indicates a padding flag, and the size of the field P is 1 bit. One or more padding bytes are included at the end of the data packet. The padding bits indicate that the data packet has been padded beyond its natural size. When the value of the field P is set to 1, it indicates that there are padding bits after the data packet. When the value of the field P is set to 0, it indicates that there are padding bits after the data packet. The last byte in the padding byte is the padding bit count, which indicates how many padding bits have been added in total. This application does not improve or expand on the field P.

[0199] Field subtype (ST): The size of the field subtype is 5 bits to allow a group of APP data packets to be defined with a unique name, or for any defined data related to the application. When Bit7=1, the receiving end requests the sending end to change the screen projection key capability set; when Bit7=0, the sending end responds to the receiving end with the changed screen projection key capability set. Bit7-Bit3 are reserved. This application has an extension to the field PT, and the extension part includes a request to change the screen projection extended capability set.

[0200] Packet Type (PT) field: When the PT field is the constant 204, it is identified as an RTCP APP packet. The PT field indicates the type of message and is 8 bits long. The RTP specification defines five standard message types, but the RTP specification is not limited to these five standard message types and can also use five other standard message types. When the PT field has a value of 204, it indicates that the message format is APP format. This application does not improve or expand the PT field.

[0201] The Synchronization Source Identifier (SSRC) field is 2 bytes long. It identifies a synchronization source. This identifier is randomly selected, but to ensure that any two synchronization source identifiers in the same RTP session are unique, RTP must detect and resolve conflicts. This application does not improve or extend the SSRC field.

[0202] The field name (name) represents the name customized by the application. This field is a name chosen by the person who defines the APP data package. The application creator may choose to use the application name and then coordinate the assignment of subtypes for other applications that want to define new data package types. In addition, the name is interpreted as a sequence of four ASCII characters, with uppercase and lowercase characters treated differently. The UI of this application has an extension with the field name (name), and the field name (name) includes the screen casting service (Cast Plus).

[0203] Application-dependent data (A): This field identifies the contents of the data packet and is 64 bits long. The extended content of the application-dependent data A field includes color: color mode: SRGB / P3 / Aodble; color range: full / limited; color primaries: BT.601PAL; transmission characteristics: BT.601; matrix system: BT.470. Basic image format: encoding method: H.264 / AVC; H.265 / HEVC; H.266P8 / P9; frame rate: 25 / 30fps / 60fps / 90fps / 120fps.

[0204] Bitrate: xxMbps; Frame Interval: 1.0 / 2.0, etc.; GOP: i-frame-interval * frame-rate. Image Quality: maxQP = 51, minQP = 12. Display Size: Height = 2400; Width = 1080.

[0205] Field Length Field (Length): refers to the length of the header and data in bytes, and the total length field is 16 bits. Each data link layer has its own frame format, which includes the maximum length of the data field in the frame format, which is called the Maximum Transmission Unit (MTU). When an IP datagram is encapsulated into a link layer frame, the total length of this datagram cannot exceed the corresponding MTU value. If the datagram length exceeds the MTU value, the datagram is fragmented. At this time, the "Total Length" field in the datagram header refers to the sum of the header length and data length of each fragment after fragmentation. This application does not improve or expand the field length field (Length).

[0206] Synchronization Source (SSRC) field: The synchronization source field is 4 bytes long and contains a synchronization source identifier, which is used to identify a synchronization source. This identifier is randomly selected, but it must be unique for any two SSRCs in the same RTP session. The sender's synchronization source identifier must be the same as the synchronization source identifier in the corresponding RTP packet.

[0207] First, the hardware structure of the electronic device 100 is introduced.

[0208] Figure 6 A schematic structural diagram of the electronic device 100 is shown.

[0209] The electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, an in-vehicle device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0210] The electronic device 100 may include a processor 110, an external memory interface 121, an internal memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0211] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0212] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0213] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0214] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0215] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0216] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0217] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0218] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0219] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0220] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0221] The wireless communication module 160 can be used to establish a communication connection (e.g., a Wi-Fi direct communication connection, a Bluetooth communication connection, etc.) with the large-screen device 200, and based on the communication connection, encode the content displayed on the user interface of the electronic device 100 and send it to the large-screen device 200. That is, the wireless communication module 160 can support content sharing between the electronic device 100 and the large-screen device 200 based on collaborative screen projection (e.g., miracast).

[0222] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0223] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0224] In the embodiment of the present application, the display screen 194 is used to display the user interface mentioned in the embodiment of the present application implemented on the electronic device 100. The specific implementation of the user interface can refer to the relevant description of the subsequent method embodiment.

[0225] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0226] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0227] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0228] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0229] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0230] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0231] The internal memory 120 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0232] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation of DDR SDRAM is generally referred to as DDR5 SDRAM), etc.

[0233] Non-volatile memory may include disk storage devices and flash memory.

[0234] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage cell potential level; universal flash storage (UFS) and embedded multi media card (eMMC) can be divided into UFS and embedded multi media card according to the storage specification.

[0235] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0236] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0237] The external memory interface 121 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 via the external memory interface 121 to implement data storage functions. For example, files such as music and videos can be stored in the external non-volatile memory.

[0238] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0239] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0240] The speaker 170A, also called a "speaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0241] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be heard by placing the receiver 170B close to the human ear.

[0242] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to realize sound signal collection, noise reduction, and identification of sound sources, and realize directional recording function, etc.

[0243] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0244] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be located on display screen 194. There are many types of pressure sensors 180A, such as resistive, inductive, and capacitive. A capacitive pressure sensor can include at least two parallel plates made of conductive material. When force acts on pressure sensor 180A, the capacitance between the electrodes changes. Electronic device 100 determines the intensity of the pressure based on this change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch intensity based on pressure sensor 180A. Electronic device 100 can also calculate the touch location based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch location but with different touch intensities can correspond to different operation instructions. For example, when a touch operation with an intensity less than a first pressure threshold is applied to a short message application icon, a command to view short messages is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to a short message application icon, a command to create a new short message is executed.

[0245] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0246] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude using the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation.

[0247] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. Based on the detected opening and closing status of the case or flip cover, features such as automatic unlocking of the flip cover can be configured.

[0248] Accelerometer 180E can detect the magnitude of acceleration of electronic device 100 in all directions (generally three axes). It can also detect the magnitude and direction of gravity when electronic device 100 is stationary. It can also be used to identify the electronic device's posture, enabling applications such as switching between landscape and portrait modes and pedometers.

[0249] The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure distance to achieve fast focusing.

[0250] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0251] Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 based on the perceived ambient light. Ambient light sensor 180L can also be used to automatically adjust white balance when taking photos. Ambient light sensor 180L can also work with proximity light sensor 180G to detect whether electronic device 100 is in a pocket to prevent accidental touches.

[0252] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0253] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from shutting down abnormally due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to prevent abnormal shutdown due to low temperature.

[0254] The touch sensor 180K is also referred to as a "touch device." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also referred to as a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to an application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a location different from that of the display screen 194.

[0255] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals from the vibrating bones of the human body. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure pulse signals. In some embodiments, the bone conduction sensor 180M can also be set in headphones to form bone conduction headphones. The audio module 170 can parse out voice signals based on the vibration signals of the vibrating bones of the human body obtained by the bone conduction sensor 180M to implement voice functions. The application processor can parse heart rate information based on the blood pressure pulse signals obtained by the bone conduction sensor 180M to implement heart rate detection functions.

[0256] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0257] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0258] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0259] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by inserting it into or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0260] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0261] Figure 7 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0262] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0263] The application layer can include a series of application packages.

[0264] like Figure 7 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0265] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0266] like Figure 7 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0267] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0268] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0269] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0270] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0271] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0272] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0273] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0274] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0275] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0276] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0277] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0278] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0279] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0280] A 2D graphics engine is a drawing engine for 2D drawings.

[0281] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0282] It needs to be emphasized that Figure 7 This is just an illustrative example; the software structure of the electronic device 100 provided in the embodiment of the present application may also adopt other software architectures, such as Software architecture of Linux or other operating systems.

[0283] For example, Figure 8 2 shows the hardware structure of the large screen device 200. Figure 8 As shown, the large-screen device 200 may include: a video codec 221, a processor 222, a memory 223, a wireless communication processing module 224, a power switch 225, a wired LAN communication processing module 226, a high-definition multimedia interface (HDMI) communication processing module 227, a USB communication processing module 228, a display 229, and an audio module 230. The various modules may be connected via a bus. Among them:

[0284] The processor 222 can be used to read and execute computer-readable instructions. In a specific implementation, the processor 222 may primarily include a controller, an arithmetic unit (ALU), and registers. The controller is primarily responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The ALU is primarily responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and may also perform address operations and conversions. The registers are primarily responsible for storing register operands and intermediate operation results temporarily stored during instruction execution. In a specific implementation, the hardware architecture of the processor 222 may be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture.

[0285] The wireless communication processing module 224 may include a WLAN communication processing module 224A, and may also include a Bluetooth (BT) communication processing module 224B, an NFC processing module 224C, a cellular mobile communication processing module (not shown), and the like.

[0286] In some embodiments, the wireless communication processing module 224 may be configured to establish a communication connection with the electronic device 100 and receive encoded data sent by the electronic device 100 based on the communication connection. For example, the WLAN communication processing module 224A may be configured to establish a Wi-Fi direct communication connection with the electronic device 100, the Bluetooth (BT) communication processing module 224B may be configured to establish a Bluetooth communication connection with the electronic device 100, and the NFC processing module 224C may be configured to establish an NFC connection with the electronic device 100. That is, the wireless communication processing module 224 may support content sharing between the electronic device 100 and the large-screen device 200 through collaborative screen projection.

[0287] In one embodiment, the wireless communication processing module 224 can monitor signals transmitted by the electronic device 100, such as detection requests and scanning signals, discover the electronic device 100, and establish a communication connection with the electronic device 100. In another embodiment, the wireless communication processing module 224 can also transmit signals, such as detection requests and scanning signals, so that the large-screen device 200 can discover the electronic device 100 and establish a communication connection (such as a Wi-Fi P2P connection) with the electronic device 100.

[0288] In some embodiments, when the electronic device 100 and the large-screen device 200 share content through collaborative screen projection, the wireless communication processing module 224 (such as the WLAN communication processing module 224A) can also receive a scene notification from the electronic device 100. The processor 222 can parse and learn the scene, and adaptively select a playback strategy corresponding to the scene, and use the playback strategy to call the display screen 229, audio module 230 and other modules to play the content sent by the electronic device 100.

[0289] The video codec 221 is used to compress or decompress digital video. In this embodiment of the present application, the video codec 221 can decompress the screen content from the electronic device 100. The large-screen device 200 can support one or more video codecs and can play videos in one or more encoding formats. For example: MPEG1, MPEG2, MPEG3, MPEG4, etc.

[0290] The processor 222 can be used to parse signals received by the wireless communication processing module 224, such as a probe request broadcast by the large-screen device 200. The processor 222 can be used to perform corresponding processing operations based on the parsing results, such as generating a probe response, etc. The processor 222 can be used to drive the display screen 229 to perform display based on the decompression result of the video codec 221.

[0291] The memory 223 is coupled to the processor 222 and is used to store various software programs and / or multiple sets of instructions. In a specific implementation, the memory 223 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 223 can store an operating system, such as an embedded operating system such as uCOS, VxWorks, RTLinux, Harmony, Android, etc. The memory 223 can also store a communication program that can be used to communicate with the large-screen device 200, one or more servers, or additional devices.

[0292] The power switch 225 can be used to control the power supply to the large-screen device 200.

[0293] The wired LAN communication processing module 226 may be used to communicate with other devices in the same LAN via the wired LAN, and may also be used to connect to a WAN via the wired LAN and communicate with devices in the WAN.

[0294] The HDMI communication processing module 227 may be configured to communicate with other devices via an HDMI interface (not shown).

[0295] The USB communication processing module 228 may be used to communicate with other devices via a USB interface (not shown).

[0296] In some embodiments, the large-screen device 200 can be connected to the display screen 300 via a USB interface, and the large-screen device 200 can transmit the displayed projection image frame to the display screen 300.

[0297] The display screen 229 can be used to display images, videos, etc. The display screen 229 can be LCD, OLED, AMOLED, FLED, QLED, etc. The content displayed on the display screen 229 can refer to the relevant description of the subsequent method embodiments.

[0298] The audio module 230 can be used to output audio signals through the audio output interface, so that the large-screen device 200 can support audio playback. The audio module 230 can also be used to receive audio data through the audio input interface. The audio module 230 may include but is not limited to: a microphone, a speaker, a receiver, etc.

[0299] In some embodiments, the large-screen device 200 may further include a serial interface such as an RS-232 interface. The serial interface can be connected to other devices, such as an audio player such as a speaker, so that the display and the audio player can play audio and video together.

[0300] It is understandable that Figure 8 The illustrated structure does not constitute a specific limitation on the large-screen device 200. In other embodiments of the present application, the large-screen device 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0301] The software system of the large screen device 200 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. For example, the software system of the large screen device 200 includes but is not limited to Linux or other operating systems. It is Huawei's Hongmeng system.

[0302] For example, Figure 9 3 shows the hardware structure of the display device 300. Figure 9 As shown, the display device 300 may include: a video codec 321, a processor 322, a memory 323, a wireless communication processing module 324, a power switch 325, a wired LAN communication processing module 326, a high-definition multimedia interface (HDMI) communication processing module 327, a USB communication processing module 328, and a display screen 329. The various modules may be connected via a bus.

[0303] The processor 322 can be used to read and execute computer-readable instructions. In a specific implementation, the processor 322 may primarily include a controller, an arithmetic unit (ALU), and registers. The controller is primarily responsible for decoding instructions and issuing control signals for operations corresponding to the instructions. The ALU is primarily responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and may also perform address operations and conversions. The registers are primarily responsible for storing register operands and intermediate operation results temporarily stored during instruction execution. In a specific implementation, the hardware architecture of the processor 322 may be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture.

[0304] The wireless communication processing module 324 may include a WLAN communication processing module 324A, and may also include a Bluetooth (BT) communication processing module 324B, an NFC processing module 324C, a cellular mobile communication processing module (not shown), and the like.

[0305] In some embodiments, the wireless communication processing module 324 may be configured to establish a communication connection with the large-screen device 200 and receive, based on the communication connection, encoded data sent by the large-screen device 200. For example, the WLAN communication processing module 324A may be configured to establish a Wi-Fi direct communication connection with the large-screen device 200, the Bluetooth (BT) communication processing module 324B may be configured to establish a Bluetooth communication connection with the large-screen device 200, and the NFC processing module 324C may be configured to establish an NFC connection with the large-screen device 200.

[0306] In one embodiment, the wireless communication processing module 324 can monitor signals transmitted by the large-screen device 200, such as detection requests and scanning signals, discover the large-screen device 200, and establish a communication connection with the large-screen device 200. In another embodiment, the wireless communication processing module 324 can also transmit signals, such as detection requests and scanning signals, so that the display device 300 can discover the large-screen device 200 and establish a communication connection (such as a Wi-Fi P2P connection) with the large-screen device 200.

[0307] The video codec 321 is used to compress or decompress digital video. In this embodiment of the present application, the video codec 321 can decompress the screen content from the large-screen device 200. The display device 300 can support one or more video codecs and can play videos in one or more encoding formats. For example: MPEG1, MPEG2, MPEG3, MPEG4, etc.

[0308] The processor 322 can be used to parse signals received by the wireless communication processing module 324, such as a probe request broadcast by the display device 300. The processor 322 can be used to perform corresponding processing operations based on the parsing results, such as generating a probe response. The processor 322 can also be used to drive the display screen 329 to perform display based on the decompression results of the video codec 321.

[0309] The memory 323 is coupled to the processor 322 and is used to store various software programs and / or multiple sets of instructions. In a specific implementation, the memory 323 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 323 can store an operating system, such as an embedded operating system such as uCOS, VxWorks, RTLinux, Harmony, Android, etc. The memory 323 can also store a communication program that can be used to communicate with the display device 300, one or more servers, or an additional device.

[0310] The power switch 325 may be used to control the supply of power to the display device 300 .

[0311] The wired LAN communication processing module 326 may be used to communicate with other devices in the same LAN via the wired LAN, and may also be used to connect to a WAN via the wired LAN and communicate with devices in the WAN.

[0312] The HDMI communication processing module 327 may be used to communicate with other devices via an HDMI interface (not shown).

[0313] The USB communication processing module 328 may be used to communicate with other devices via a USB interface (not shown).

[0314] The display screen 329 can be used to display images, videos, etc. The display screen 329 can be LCD, OLED, AMOLED, FLED, QLED, etc. The content displayed on the display screen 329 can refer to the relevant description of the subsequent method embodiments.

[0315] In some embodiments, the display device 300 may further include a serial interface such as an RS-232 interface, which can be connected to other devices, such as an audio player such as a speaker, so that the display and the audio player can play audio and video together.

[0316] It is understandable that Figure 9 The illustrated structure does not constitute a specific limitation on the display device 300. In other embodiments of the present application, the display device 300 may include more or fewer components than shown, or may combine or separate certain components, or may have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0317] The software system of the display device 300 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. For example, the software system of the display device 300 includes but is not limited to Linux or other operating systems. It is Huawei's Hongmeng system.

[0318] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A screen projection system, characterized in that: The system includes an electronic device, a large-screen device, and a display device; the large-screen device is used to: Establishing a first screen projection connection with the electronic device and a second screen projection connection with the display device, wherein the first screen projection connection is used for the large-screen device to receive and display the projection content sent by the electronic device; The electronic device is configured to send a first display parameter to the large-screen device; wherein the first display parameter includes a first encoding frame rate of the electronic device; The display device is configured to send a second display parameter to the large-screen device; wherein the second display parameter includes a first decoding frame rate and a first display frame rate of the display device; The large-screen device is also used for: receiving the first display parameter sent by the electronic device; receiving the second display parameter sent by the display device; Acquire a third display parameter of the large-screen device, where the third display parameter includes a second encoding frame rate, a second decoding frame rate, and a second display frame rate; Based on the first display parameter, the second display parameter and the third display parameter, the target display parameters are determined, wherein the target display parameters include a target encoding frame rate, and the target encoding frame rate is less than or equal to the minimum value of the first encoding frame rate, the first decoding frame rate, the first display frame rate, the second encoding frame rate, the second decoding frame rate, and the second display frame rate; based on the target encoding frame rate in the target display parameters, the first projection content is encoded, and the first projection content is projected to the display device.

2. The system according to claim 1, wherein: Before the large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameter and projects the first projection content to the display device, the large-screen device is further configured to: Sending the target display parameter to the electronic device, wherein the target display parameter is used to instruct the electronic device to encode the second projection content at the target encoding frame rate and send it to the large-screen device; The electronic device is further used for: Encoding a second projection content at the target encoding frame rate; and sending the second projection content to the large-screen device; The large-screen device is further used for: Receive the second projection content sent by the electronic device; and display the second projection content.

3. The system according to claim 1, wherein: The large-screen device is specifically used for: After the large-screen device establishes the first screen projection connection with the electronic device, the second screen projection connection is established with the display device; or, After the large-screen device establishes the second screen projection connection with the display device, the first screen projection connection is established with the electronic device.

4. The system according to claim 1, wherein: Before the large-screen device receives the first display parameter sent by the electronic device, the large-screen device is further configured to: Sending a first request to the electronic device, where the first request is used to instruct the electronic device to send the first display parameter to the large-screen device; The electronic device is further used for: receiving the first request sent by the large-screen device; In response to the first request, sending the first display parameter to the large-screen device; The large-screen device is also used for: Receive the first display parameter sent by the electronic device.

5. The system according to claim 4, characterized in that The large-screen device is specifically used for: When it is determined that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, the first request is sent to the electronic device.

6. The system according to any one of claims 1 to 5, characterized in that The first display parameter further includes a first encoding type and / or a first screen resolution; The second display parameters further include a second encoding type and / or a second screen resolution; The target display parameters also include target encoding type and / or target screen resolution; The target encoding type is a common encoding type of the first encoding type and the second encoding type, and the target screen resolution is a common screen resolution of the first screen resolution and the second screen resolution.

7. A method for adjusting screen projection display parameters, characterized in that: Methods include: The large-screen device establishes a first screen projection connection with the electronic device, and the large-screen device establishes a second screen projection connection with the display device, wherein the first screen projection connection is used for the large-screen device to receive and display the projection content sent by the electronic device; The large-screen device receives a first display parameter sent by the electronic device, where the first display parameter includes a first encoding frame rate of the electronic device; The large-screen device receives a second display parameter sent by the display device, where the second display parameter includes a first decoding frame rate and a first display frame rate of the display device; The large-screen device obtains a third display parameter of the large-screen device, where the third display parameter includes a second encoding frame rate, a second decoding frame rate, and a second display frame rate; The large-screen device determines a target display parameter based on the first display parameter, the second display parameter, and the third display parameter, wherein the target display parameter includes a target encoding frame rate, and the target encoding frame rate is less than or equal to a minimum value among the first encoding frame rate, the first decoding frame rate, the first display frame rate, the second encoding frame rate, the second decoding frame rate, and the second display frame rate; The large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameters, and projects the first projection content to the display device.

8. The method according to claim 7, characterized in that Before the large-screen device encodes the first projection content based on the target encoding frame rate in the target display parameter and projects the first projection content to the display device, the method further includes: The large-screen device sends the target display parameter to the electronic device, wherein the target display parameter is used to instruct the electronic device to encode the second projection content at the target encoding frame rate and send it to the large-screen device; The large-screen device receives the second screen projection content sent by the electronic device; The large-screen device displays the second projection content.

9. The method according to claim 7, characterized in that The large-screen device establishes a first screen projection connection with the electronic device, and the large-screen device establishes a second screen projection connection with the display device, specifically including: After the large-screen device establishes the first screen projection connection with the electronic device, the large-screen device establishes the second screen projection connection with the display device; or, After the large-screen device establishes the second screen projection connection with the display device, the large-screen device establishes the first screen projection connection with the electronic device.

10. The method according to claim 7, characterized in that Before the large-screen device receives the first display parameter sent by the electronic device, the method further includes: The large-screen device sends a first request to the electronic device, where the first request is used to instruct the electronic device to send the first display parameter to the large-screen device.

11. The method according to claim 10, characterized in that The large-screen device sending a first request to the electronic device specifically includes: When the large-screen device determines that the second encoding frame rate is greater than the first decoding frame rate or the second encoding frame rate is greater than the first display frame rate or the second encoding frame rate is less than the second display frame rate, the large-screen device sends the first request to the electronic device.

12. The method according to any one of claims 7 to 11, characterized in that: The first display parameter further includes a first encoding type and / or a first screen resolution; The second display parameters further include a second encoding type and / or a second screen resolution; The third display parameter further includes a third encoding type and / or a third screen resolution; The target display parameters also include target encoding type and / or target screen resolution; The target encoding type is a common encoding type of the first encoding type and the second encoding type, and the target screen resolution is a common screen resolution of the first screen resolution and the second screen resolution.

13. An electronic device, characterized in that: The electronic device comprises one or more processors and one or more memories; the one or more memories and the one or more encoders are coupled to the one or more processors, the one or more memories are used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 7 to 12.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to execute the method according to any one of claims 7 to 12 when called by the computer.

15. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to perform the method according to any one of claims 7 to 12.

Images